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Abstract 


A  validation  assessment  of  THUNDER  6.5’s  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  module  is  accomplished  using  formulational  and  experimental 
validation  techniques.  A  comparison  of  ISR  purposes  and  processes  according  to 
military  doctrine  is  made  with  the  purposes  and  processes  of  ISR  implemented  within 
THUNDER  6.5.  This  comparison  provides  an  overview  of  the  process,  an  understanding 
of  the  level  of  aggregation  within  THUNDER,  insight  into  possible  problem  areas  in 
THUNDER,  and  a  basis  for  improving  THUNDER  ISR  processes.  Sensitivity  analysis  of 
the  ISR  parameters  as  they  relate  to  the  Quality,  Quantity,  and  Timeliness  of  ISR  is  also 
presented  to  provide  insight  into  the  responsiveness  of  THUNDER  to  changes  in  ISR 
capability  for  selected  battle  outcomes.  Linear  Regression  and  a  Face-Centered  Central 
Composite  Design  were  used  to  generate  a  response  surface.  Ninety-percent  confidence 
intervals  were  used  to  determine  differences  in  mean  response  among  the  fall  factorial 
design  points. 
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A  VALIDATION  ASSESSMENT  OF  THUNDER  6.5’S  INTELLIGENCE, 
SURVEILLANCE,  AND  RECONNAISSANCE  MODULE 


1.  Introduction 


1.1  Combat  Modeling  Tools 

The  modeling  and  simulation  community  employs  a  multitude  of  models  that 
differ  according  to  their  intended  purpose.  These  models/simulations  are  grouped 
together  into  categories  -  engineering,  engagement,  mission,  and  campaign  -  according  to 
their  resolution,  quantitative,  and  qualitative  characteristics  (see  Figure  1).  The  numerous 
models  and  the  degrees  of  resolution  are  necessary  to  accommodate  the  wide  range  of 
users  and  their  fundamentally  different  information  requirements  [5]. 


Campaign 


Mission 


Engagement 


Engineering 


Figure  1 .  Air  Force  Analytical  Toolkit  Example  [53] 
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The  quantitative  differences  among  models  involve  scope,  scale,  duration,  and 
aggregation  [5].  Each  of  these  factors  increases  moving  up  the  pyramid.  The  scope  of  an 
engineering  model  is  limited  to  one  system  or  subpart,  whereas  the  scope  of  a  campaign- 
level  model  includes  the  whole  theater  of  war.  The  scale  progresses  up  the  pyramid  from 
representing  components  of  a  system  at  the  engineering-level  to  combat  engagements  of 
“many-on-many”  in  campaign-level  models.  The  duration  used  in  engagement  models 
focuses  on  discrete  times  or  events  such  as  target  acquisition,  whereas  campaign  models 
include  many  operations  and  can  simulate  a  whole  war.  The  aggregation  level  of 
components  also  increases  from  engineering  to  campaign-level  models.  In  fact, 
campaign-level  models  aggregate  the  output  of  engineering/engagement  model  and  use 
this  aggregated  output  as  input  data.  Figure  2  displays  the  differences  among  model 
resolution  and  interactions. 
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Figure  2.  Comparison  of  Model  Resolutions  [5] 
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Qualitative  differences  also  exist  among  the  models.  These  differences  include 
human  behavior  representation,  measures  of  effectiveness  employed,  and  use  of  the 
output.  Human  decision-making  is  seen  in  engagement  and  mission-level  models.  It  is 
not  found  in  engineering  models  since  they  are  concerned  with  physical  processes,  and 
campaign  level  models  exclude  it  mostly  due  to  the  large  number  of  decision  points  and 
choices  that  can  make  it  unmanageable  [5].  The  measures  of  effectiveness  change  as  one 
moves  up  the  pyramid.  An  engineering-level  model  measures  its  output  in  scientific 
terms  such  as  amps,  whereas  campaign  level  models  must  relate  their  output  to  a 
warfighting  context  such  as  days  to  achieve  air  superiority  [5].  How  the  output  of  the 
model  is  used  also  differs  among  the  levels  of  the  pyramid.  As  the  quantitative  and 
explicit  representation  of  elements  decreases,  models  become  less  useful  in  predictive 
capability  and  become  more  useful  for  simply  providing  insight  into  a  system’s  influence 
on  the  warfighting  scenario.  “Given  [the]  level  of  complexity,  the  results  of  campaign 
simulations  should  be  seen  as  less  an  ‘answer’  than  an  insight,  less  predictive  than 
indicative”  [5]. 

THUNDER  is  positioned  in  the  Air  Force’s  Analytical  Toolkit  (Figure  1)  at  the 
campaign-level  and  is  considered  the  Air  Force’s  premier  campaign-level  model. 
THUNDER  is  an  “...analytical  tool  designed  to  help  senior  decision-makers  evaluate 
strategy,  tactics,  force  structure,  and  operational  effectiveness  in  a  joint  warfighting 
context”  [47].  It  is  an  important  tool  in  evaluating  the  impact  of  proposed  weapon 
systems,  technology,  tactics,  and  doctrine  on  combat  outcomes. 
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1.2  Problem  Statement 


Intelligence  is  critical  to  the  success  of  military  operations.  Timely,  accurate 
intelligence  can  be  the  defining  element  between  victory  and  defeat.  This  aspect  of  war 
must  be  incorporated  into  combat  models  to  ensure  the  model  is  an  effective  tool  and  that 
combat  is  correctly  represented. 

THUNDER  has  integrated  intelligence,  surveillance,  and  reconnaissance  (ISR) 
representation  into  its  combat  processes.  The  ISR  module  of  THUNDER  affects  many 
aspects  of  the  war,  including  prioritization  of  targets,  aircraft/munition  selection,  target 
acquisition,  and  ground  unit  attrition.  Since  the  complex  and  compounding  effects  of 
ISR  significantly  impact  campaign  outcomes  in  the  real  world,  one  must  ensure  that  the 
impact  of  ISR  is  accurately  represented  in  THUNDER.  Ensuring  a  model  accurately 
reflects  a  real-world  process  is  accomplished  through  validation.  The  purpose  of  this 
thesis  is  to  assess  the  validity  of  the  ISR  module  of  THUNDER  6.5. 

1.3  Validation  of  Models 

Validation  is  defined  by  Joint  Pub  1-02,  Department  of  Defense  Dictionary  of 
Military  and  Associated  Terms,  as  “The  process  of  determining  the  degree  to  which  a 
model  or  simulation  is  an  accurate  representation  of  the  real  world  from  the  perspective 
of  the  intended  uses  of  the  model  or  simulation”  [9:488]. 

Campaign-level  models  are  very  difficult  to  validate  due  to  the  complexity  of 
factors  and  the  lack  of  real  world  data.  Also,  because  models  like  THUNDER  are  used 
for  analyses  on  proposed  future  system,  validating  outcomes  is  often  impossible.  One 
inherent  problem  with  validation  is  the  definition  of  reality.  The  most  common 
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references  for  establishing  validity  are  field  experts  and  historical  data.  For  this  study, 
reality  is  defined  by  current  military  doctrine  and  instructions  about  ISR. 

This  thesis  used  two  approaches  to  assess  the  validity  of  THUNDER  6.5’s  ISR 
module.  The  first  approach  examines  the  real  world  ISR  process  and  to  compares  it  with 
the  ISR  process  within  THUNDER,  and  the  second  is  to  perform  an  experiment  to  verify 
the  sensitivity  of  the  ISR  processes  on  combat  outcomes.  Both  of  these  approaches  are 
necessary  for  validation,  but  neither  one  alone  is  sufficient.  Comparative  analysis 
between  competing  systems  is  a  primary  use  of  campaign  models.  A  model  that 
represents  the  parameters  well,  but  lacks  in  providing  representative  output  has  limited 
use.  On  the  other  hand,  a  model  that  provides  correct  output,  but  does  not  represent  the 
parameters  well  may  be  unable  to  provide  insight  if,  or  when,  reality  changes  or  a  new 
scenario  is  introduced. 

These  two  approaches  also  fit  into  the  “four  major  interdependent  types  of 
validation”  presented  by  Oral  and  Ketanni:  formulational,  experimental,  operational,  and 
data  validation  [32:223]. 

Comparing  the  processes  provides  for  formulational  validation  as  it  is  “...mainly 
concerned  with  the  degree  of  capacity  of  the  ‘formal  model’  to  describe  correctly  and 
accurately  [real  world  events]”  [32:224].  Experimental  validation  primarily  deals  with 
“...the  quality  of  solutions,  the  types  of  solutions,  the  nature  of  solution  techniques,  and 
the  efficiency  of  solution  procedures”  [32:224].  The  experiment  performed  in  this  study 
focuses  on  the  quality  of  solutions  as  it  relates  to  the  level  of  insight  gained  about  the 
warfighting  scenario  and  the  sensitivity  to  changes  in  the  values  of  the  ISR  parameters. 
Operational  validation  which  refers  to  the  “...usability,  usefulness,  timeliness,  synergism. 
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and  the  cost  of  implementing  a  decision  based  on  the  solutions  provided  by  the  ‘formal 
model’”  [32:226]  is  beyond  the  scope  of  this  thesis.  Data  validation  involving 
“...sufficiency,  accuracy,  appropriateness,  availability,  maintainability,  reliability,  and 
cost  of  data”  [32:222]  is  not  relevant  to  this  thesis. 

1.4  Modeling  and  Validating  Intelligence 

THUNDER’S  ISR  module  can  be  used  to  evaluate  intelligence  systems  in  terms  of 
their  contributions  to  combat  outcomes.  This  evaluation  allows  for  comparisons  between 
systems  and  their  capabilities  as  well  as  the  combined  value  of  a  group  of  systems. 

Intelligence  is  difficult  to  measure,  and  difficult  to  validate  because  of  the 
uncertainty  it  adds  to  combat  situations.  A  RAND  study  [2]  on  assessing  the  combat 
value  of  intelligence  identifies  two  main  reasons  why  modeling  intelligence  systems 
largely  increases  the  uncertainty  that  already  exists  in  a  model.  These  reasons  are  soft 
(human)  factors  and  nonlinearity. 

Intelligence  produces  information.  Information  influences  human  decisions. 
Modeling  a  human  decision  process  is  difficult,  hard  to  represent,  hard  to  calibrate,  and 
hard  to  validate;  therefore,  it  is  often  poorly  reflected  in  modeling  [2:2],  “This  neglect  is 
sometimes  justified  either  because  human  factors  are  believed  to  be  less  critical  than 
‘hard’  technical  characteristics  or  because  there  are  so  many  human  actors  involved  that 
their  actions  can  be  represented  statistically  by  aggregate  probabilities,  depending  on  the 
application”  [2:2].  Combat  elements  such  as  target  acquisition  can  be  modeled  with  these 
hard  characteristics,  however,  elements  such  as  situation  assessment  require  the 
commander’s  decision  process  to  be  modeled.  “Focusing  only  upon  target  acquisition 
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would  provide  a  more  tractable  problem  but  would  systematically  underestimate  the 
value  of  [intelligence]”  [2:2]. 

Intelligence  has  a  highly  nonlinear  effect  on  battle.  In  and  of  itself,  intelligence  is 
of  no  value,  but  its  effectiveness  is  seen  through  the  enhancement  of  other  combat 
elements.  The  effect  of  intelligence  can  be  largely  dependent  on  the  situation  and 
decision  at  hand.  For  example,  if  an  aircraft  has  expended  its  munitions,  knowing  where 
the  next  target  is  with  a  high  degree  of  accuracy  is  of  no  consequence.  However,  when  a 
“...single  command  decision  means  the  difference  between  victory  and  defeat  (for 
example  deciding  when  to  commit  strategic  reserves), . .  .the  effect  of  one  piece  of  critical 
intelligence  is  so  nonlinear  as  to  be  essentially  discontinuous”  [2:3].  This  fact  increases 
the  uncertainty.  Small  changes  in  input  can  result  in  large  changes  in  output. 

Although  the  difficulty  in  modeling  ISR  is  great,  the  importance  of  ISR  and  its 
effects  on  combat  outcomes  make  it  an  essential  element  that  cannot  be  omitted  if  a 
model  is  to  truly  represent  combat  operations. 

1.5  Thesis  Outline 

This  thesis  is  organized  into  chapters  according  to  subject  areas.  Chapter  2 
presents  an  overview  of  THUNDER,  some  of  its  components,  users,  and  history.  Chapter 
3  provides  information  on  current  military  doctrine  concerning  the  purposes  and 
principles  of  ISR.  Chapter  4  examines  the  ISR  process  and  its  elements.  The  objective 
of  Chapters  3  and  4  is  to  demonstrate  the  complexity  and  breadth  of  the  ISR  process  in 
the  real  world.  This  complex,  sometimes  unquantifiable,  process  has  to  be  resolved  into 
something  that  can  be  represented  by  a  model,  and  Chapter  5  explains  how  THUNDER 
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currently  accomplishes  that  goal.  The  comparison  between  the  ISR  purposes  and 
processes  in  the  real  world  and  in  THUNDER  is  presented  in  Chapter  6.  This 
comparison  provides  an  overview  of  the  process,  an  understanding  of  the  level  of 
aggregation  within  THUNDER,  insight  into  possible  problem  areas  in  THUNDER,  and  a 
basis  for  improving  THUNDER  ISR  processes.  The  scope  of  the  comparison  is  reserved 
to  major  steps  in  the  processes,  with  some  discussion  of  lower  details  to  analyze  whether 
the  aggregation  within  THUNDER  is  appropriate.  Chapter  7  examines  how  THUNDER 
reacts  to  changes  in  ISR  capability.  Sensitivity  analysis  of  parameters  relating  to  the 
quality,  quantity,  and  timeliness  of  ISR  provides  insight  into  the  responsiveness  of 
THUNDER  to  the  ISR  module.  The  final  chapter,  Chapter  8,  presents  the  conclusions  of 
this  thesis  and  recommendations  for  improving  THUNDER’S  ISR  module. 

Throughout  this  thesis,  the  term  THUNDER  refers  to  the  THUNDER  version  6.5. 
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2.  THUNDER  Overview 


2.1  Introduction 

THUNDER  is  a  two-sided,  stochastic,  campaign-level  model  of  conventional  air 
and  land  warfare,  with  some  naval  representation.  It  is  the  campaign-level  model  of 
choice  for  the  Air  Force  that  has  been  used  in  high  level  decision-making  activities  such 
as  determining  the  advantages  of  the  F-22  and  the  effect  of  the  Global  Positioning  System 
(GPS)  on  the  warfighter.  This  chapter  presents  a  brief  overview  of  the  model,  its  users, 
history,  and  verification  and  validation  efforts. 

2.2  Model  Functional  Design 

The  battlefield  is  modeled  as  a  rectangular  area  which  can  be  oriented  in  any 
direction  that  the  user  specifies.  It  is  subdivided  into  grid  squares,  whose  sizes  are  also 
defined  by  the  user.  A  Forward  Line  of  Own  Troops  (FLOT)  line  spans  the  width  of  the 
battlefield  and  divides  the  two  opposing  sides,  referred  to  as  Red  and  Blue.  See  Figure  3. 

The  battlefield  is  further  divided  into  sectors  and  zones,  as  seen  in  Figure  4. 
Sectors  represent  commands  that  own  units  and  can  control  portions  of  the  FLOT.  Each 
sector  is  divided  into  zones  that  generally  represent  areas  in  which  certain  ground 
activities  take  place.  Units  that  are  engaged  in  combat  are  located  in  the  zone  adjacent  to 
the  FLOT. 

The  ISR  grid  in  THUNDER  is  similar  in  shape  and  form  to  the  battlefield  grid 
and  can  be  thought  of  as  overlaying  the  battlefield  grid. 
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The  ground  war  within  THUNDER  is  a  deterministic  model  based  on  the  United 
States  Army’s  Concept  Evaluation  Model  (CEM).  The  CEM  uses  the  Attrition 
Calibration  (ATCAL)  process  and  the  United  States  Army  Concepts  and  Analysis 
Agency’s  Combat  Sample  Generator  (COSAGE)  model  to  adjudicate  ground  combat  on 
a  cyclical  basis  defined  by  the  user.  Units  engaged  in  combat  move  strictly  back  and 
forth  (similar  to  movement  such  as  a  piston)  within  a  sector.  Theater  level  supplies  and 
logistics  for  both  air  and  ground  forces  are  modeled  using  mathematical  transportation 
networks.  Unit  consumption  data  comes  from  the  US  Army  Combined  Arms  Support 
Command  (CASCOM). 

For  the  air  war,  THUNDER  simulates  27  different  air  missions  such  as  airborne 
refueling,  close  air  support,  defensive  counter-air,  fighter  sweep,  long  range  air 
interdiction,  reconnaissance,  and  standoff  reconnaissance.  Some  of  the  missions  defined 
in  THUNDER  are  merely  subsets  of  air  missions  defined  by  Air  Force  doctrine.  These 
subsets  allow  for  finer  fidelity  to  capture  desired  effects,  as  well  as  for  modeling 
convenience.  THUNDER  “...automatically  generates  Air  Tasking  Orders  (ATOs)  based 
on  theater  level  apportionment  and  target  priorities.  A  scheduler  builds  ATO  packages, 
taking  into  consideration  aircraft  range,  weather  capability,  weapons  configurations, 
weapons  effectiveness,  weapons  availability,  and  target  availability”  [54:2]. 

THUNDER  is  data  driven  and  relies  on  a  myriad  of  engineering  and  engagement 
models  to  provide  its  data  inputs  and  calculations.  For  example,  BRAWLER  is  used 
extensively  for  air-to-air  information;  ESAMS  is  used  for  surface-to-air  missile 
probability  of  kill  (Pk)  data;  and  RADGUNS  for  anti-defense  artillery  Pks.  Some  other 
aspects  of  war  modeled  in  THUNDER  are:  airbase  operations  including  maintenance  of 
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aircraft  and  runway  repair;  cruise  missile  attacks  and  defense;  intelligence,  surveillance, 
and  reconnaissance  (ISR);  perception  and  “fog  of  war”;  strategic  attacks;  multiple-target 
missions;  and  the  effects  of  weather. 

THUNDER  is  written  in  SIMSCRIPT  II.5,  a  general-purpose  programming 
language  particularly  suited  for  large,  discrete-event  simulations.  THUNDER  contains 
about  300,000  lines  of  code  divided  into  over  1,350  routines.  It  operates  on  Sun  and 
Silicon  Graphics  Unix  workstations. 

2.3  Users 

THUNDER  is  used  by  a  large  number  of  U.S.  and  allied  defense  organizations 
and  contractors  such  as  Air  Force  Studies  and  Analyses  Information  Superiority  Branch 
(AFSAA/SAAI),  AFSAA  Wargaming  Branch  (AFSAA/SAAW),  Air  Force  Wargaming, 
HQ  ACC/XP-SAS,  Boeing  North  American,  Lockheed  Martin  Tactical  Aeronautical, 
Northrop  Grumman,  and  Raytheon  Systems  Company.  It  is  also  used  internationally  by 
British  Aerospace,  Defense  Science  and  Technology  Organisation  (Australia),  RAF  Air 
Warfare  Centre,  and  the  Republic  of  Korea  Air  Force. 

2.4  Assumptions 

Because  of  the  amount  of  aggregation  needed  in  campaign  level  models, 
numerous  assumptions  are  made  regarding  data  validity.  Users  must  understand  how 
input  data  has  been  aggregated,  and  how  the  model  uses  the  data.  Aside  from  that,  the 
three  major  assumptions  identified  for  THUNDER  for  any  campaign  being  studied  are: 

1)  The  war  is  between  two  nation-state  sized  adversaries  in  a  single  theater  of 
operations. 
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2)  A  defined  boundary  exists  between  opposing  sides  in  the  model. 

3)  The  campaign  can  be  expressed  by  a  four-part  process:  Perception,  Planning, 
Execution,  and  Adjudication. 

2.5  Origin  and  History 

THUNDER  was  originally  developed  from  TAC  WARRIOR,  a  campaign-level 
model  used  in  the  1970’s  and  early  1980’s.  As  TAC  WARRIOR  evolved,  it  became 
difficult  to  modify,  the  documentation  no  longer  matched  the  code,  and  the  assumptions 
of  the  model  became  invalid.  To  compensate  for  this,  Air  Force  Studies  and  Analyses 
developed  THUNDER  from  1983-1986.  THUNDER  achieved  its  initial  operating 
capability  in  1985,  and  the  first  operational  version  was  released  in  1986.  CACI,  Inc. 
performed  maintenance  and  upgrades  from  1987-1993.  Since  1993,  both  CACI,  Inc.  and 
System  Simulation  Solutions,  Inc.  (S3I)  have  maintained  THUNDER.  The  most  current 
version  is  THUNDER  6.5,  released  in  November  of  1997.  Table  1  shows  the  history  of 
THUNDER  model  releases  and  some  of  the  developments. 

Table  1.  THUNDER  Release  Chronology  [50:5-7,11] 


Release  Date 

Version 

Significant  modification/enhancement 

Aug  87 

3.0 

VAX/VMS  support 

CACI  Products  Company  assumes  configuration  control 

May  88 

4.0 

SUN/UNIX  support 

Jan  89 

4.4 

Situation  map 

Graphic  utility 

Nov  89 

4.6 

Enhancement  of  air  defense  module  and  ground  module 

Jun  90 

5.1 

Incorporation  of  rear-area  transportation  network 

Enhanced  Suppression  of  Enemy  Air  Defenses  functions 

Jun91 

5.5 

Enhancement  of  airfield  attack  mission 

Introduction  of  detailed  logistics  resource 
accounting  methodology 

Enhancement  of  air  mission  planning 

May  92 

5.6 

Addition  of  strategic  target  interdiction 

Enhanced  ground  attrition  methodology 
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Dec  92 

5.8 

Addition  of  high-fidelity  sortie  rate  profiles 

Addition  of  high-resolution  aircraft  maintenance  module 
Improved  methodology  for  treating 

overrun/abandoned  airbases 

Feb  93 

5.9 

Addition  of  time-dependent  aircraft  planning  factors 
Accommodation  of  multiple  target  sorties 

Sep  93 

6.0 

Incorporation  of  new  flight  missions: 

Enhanced  terminal  air  defense  logic 

May  94 

6.1 

Deterministic  weather  model 

Major  revision  of  air-to-surface  adjudication  methodology 
Enhanced/standardized  target  repair  methodology 

Enhanced  air-to-surface  targeting  prioritization 

Jun  94 

6.2 

Higher  resolution  aerial  refueling  methodology 

Enhanced  air  mission  abort  rules 

Improved  sortie  scheduling  algorithm  in  ATO  generator 
Improved  flight  path  logic 

Accommodate  survivability  in  configuration  selection 

More  flexible  SEAD  corridor  selection  logic 

Jun  95 

6.3 

Model  effects  of  integrated  air  defense  systems 

More  explicit  play  of  intelligence,  surveillance,  and 
reconnaissance  (ISR)  systems  and  effects 
Higher  fidelity  surface-to-air  lethality  data 
Accommodate  integrated  lethal/non-lethal  SEAD  mission 
Improved  terminal  delivery  profile  to  accommodate 

_ weapon  specific  delivery  requirements _ 

Sep  96  6.4  Enhanced  flight  path  generation  algorithm  to  avoid 

area  SAM  threat 

Maj  or  ISR  methodology  improvements 
Addition  of  satellites  to  ISR  module 
Incorporation  of  explicit  TBM  model  and  anti-TBM 
missions  (pre/post-launch) 

Improved  carrier  battle  group  play _ 

Nov  97  6.5  Interdictable  ISR  nodes 

Weather  effects  on  ISR 

Weapons  of  Mass  Destruction  (Chem/Bio) 

Enhanced  target  acquisition  and  discrimination 
Improved  air  network  paths 
Use  perceived  threat  in  air  network  survivability 
calculations 

Enhanced  terminal  air  defense 
Incorporate  tactical  airlift 
Rule-based  air  planning 
Cyclic  carrier  operations 
Ground  force  engagement  rules 
Grid-based  air  defense  enhancements 
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2.6  Current  Verification  and  Validation  Efforts  [54] 

Verification  of  a  model  ensures  that  the  model  runs  as  intended.  THUNDER 
conforms  to  the  industry-accepted  software  standards  for  design  and  coding.  The  coding 
standards  ensure  that  the  code  is  understandable,  promoting  efficiency  in  maintenance 
and  implementing  new  developments.  The  standards  also  give  analysts  the  ability  to 
quickly  comprehend  the  algorithms,  providing  insights  into  model  assumptions  and 
expected  outcomes.  Along  with  “easy  to  read”  code,  comments  embedded  in  the  code 
are  also  helpful  analysis  tools. 

THUNDER’S  Configuration  Management  Plan  accounts  for  version  control  and 
release  authority  through  a  Revision  Control  System,  a  Configuration  Control  Board,  and 
a  formal  release  cycle  and  version  numbering  system.  The  Revision  Control  System 
archives  changes  to  the  model.  The  Configuration  Control  Board  is  the  formal  authority 
for  reviewing  model  modifications  and  approving  model  changes. 

All  elements  of  the  core  THUNDER  model  were  verified  during  the  original 
implementation  through  requirements-to-design  tracing,  walkthroughs  and  formal 
reviews  of  the  code,  component  and  integration  testing,  and  alpha  and  beta  release  test 
phases.  These  elements  are  also  re-evaluated  during  significant  modifications  or 
enhancements  to  baseline  releases. 

Validation  of  a  model  ensures  that  the  model  accurately  reflects  the  process  or 
system  that  it  represents.  THUNDER  uses  two  methods  in  its  validation  process: 
structural  validation  and  output  validation.  In  structural  validation,  subject  matter  experts 
evaluate  the  algorithms  and  code  to  determine  if  the  implementation  of  the  model  will 
match  the  intent  of  the  programmers.  Output  validation  involves  subject  matter  experts 
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examining  results  to  determine  the  extent  that  the  implementation  of  the  model  matches 
expected  outcomes.  The  output  subject  matter  experts  involved  in  THUNDER  validation 
are  usually  members  of  the  organization  funding  a  modification. 

2.7  Summary 

THUNDER  is  an  extremely  large  and  complex  model  that  is  continually  being 
updated  to  improve  its  ability  to  model  various  and  new  aspects  of  war.  It  can  be 
cumbersome  for  the  user  as  the  model  requires  an  enormous  amount  of  data  -  almost  100 
input  files  are  required.  However,  the  usefulness  of  the  model  cannot  be  overstated.  Its 
real  value  comes  from  the  analysis  of  various  weapon  types,  capabilities,  strategies,  and 
their  interactions.  Using  point  estimates  of  model  outcomes  can  be  misleading  due  to  the 
numerous  input  data  assumptions  and  systemic  limitations.  The  high  level  of  aggregation 
in  campaign  models  is  somewhat  of  a  disadvantage,  but  not  something  resolvable. 

Careful  and  valid  data  aggregation  from  higher  resolution  models  is  needed  to  lessen  this 
disadvantage. 

THUNDER  is  considered  a  valid,  highly  effective  model  for  the  Air  Force.  Its 
strength  is  that  the  Air  War  is  modeled  at  a  slightly  higher  resolution  than  is  available 
elsewhere,  with  the  automatic  ATO  generator  saving  a  lot  of  user  input  time.  The 
Ground  War  is  deterministic,  which  saves  on  run-time,  but  it  provides  for  somewhat 
limited  interaction  between  air  and  ground  units.  Ship-to-ship  naval  warfare  is  not 
modeled,  but  that  does  not  seem  to  impair  the  model’s  usefulness. 
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3.  Intelligence.  Surveillance.  Reconnaissance  Overview 


3.1  Introduction 

Intelligence,  surveillance,  and  reconnaissance  (ISR)  operations  are  used  for 
numerous  military  purposes  and  involve  a  variety  of  agencies  and  support  systems  to 
accomplish  its  objectives.  This  chapter  presents  some  background  on  ISR,  its  purposes, 
players,  products,  and  underlying  systems. 

3.2  Definitions 

3.2.1  Intelligence,  Surveillance,  Reconnaissance 

Intelligence,  Surveillance,  and  Reconnaissance  is  defined  as  “The  capability  to 
collect,  process,  exploit  and  disseminate  accurate  and  timely  information  that  provides 
the  battlespace  awareness  necessary  to  successfully  plan  and  conduct  operations”  [15:8]. 

Intelligence  is  the  “Product  resulting  from  the  collection,  processing,  integration, 
analysis,  evaluation,  and  interpretation  of  available  information  concerning  foreign 
countries,  military  capabilities,  political  groups,  technological  developments  or  certain 
geographic  areas”  [15:8].  Intelligence  is  obtained  through  surveillance  and 
reconnaissance.  Although  surveillance  and  reconnaissance  are  similar,  they  are  distinct 
enough  to  warrant  separate  definitions. 

Surveillance  is  the  “Sustained  or  systematic  observation  of  aerospace,  surface  or 
subsurface  areas,  places,  persons,  or  things,  by  visual,  aural,  electronic,  photographic,  or 
other  means”  [15:8].  A  typical  example  of  a  platform  used  for  surveillance  is  the  E-8 
Joint  Surveillance  Target  Attack  Radar  System  (J-STARS)  which  orbits  for  a  relatively 
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long  period  of  time  and  provides  continuous  updates  on  enemy  strength,  location,  and 
movement. 

Reconnaissance  is  defined  as  “A  transitory  mission  undertaken  to  obtain,  by 
visual  observation  or  other  detection  methods,  information  about  the  activities  and 
resources  of  an  adversary  or  potential  adversary,  or  to  secure  data  concerning  the 
meteorological,  hydrographic,  or  geographic  characteristics  of  a  particular  area”  [15:8]. 
As  opposed  to  a  sustained  mission  as  in  surveillance,  reconnaissance  missions  do  not 
provide  continuous  coverage  of  an  area,  but  instead  collect  data  on  a  discrete  basis.  An 
example  of  reconnaissance  is  a  passing  satellite  taking  a  picture  of  a  Scud  missile 
launcher. 

Another  contrast  between  surveillance  and  reconnaissance  is  that  surveillance 
data  is  generally  passed  on  a  real-time  basis  and  usually  comes  from  one  source.  The 
data  provided  is  most  often  perishable  and  needed  immediately,  even  before  analysis. 
Reconnaissance  data  usually  passes  through  a  processing  phase  of  evaluation  by  analysts 
and  fusion  with  other  data  [34:14]. 

Surveillance  and  reconnaissance  resources  gather  data  that  can  be  processed, 
analyzed,  and/or  fused  to  produce  intelligence.  Fusion  is  the  process  of  combining  data 
from  various  collection  sources  to  produce  more  accurate  information  than  from  one 
source  alone.  Fusion  provides  a  more  comprehensive,  yet  focused,  intelligence  product. 

3.3  Intelligence  Purposes 

The  role  of  ISR  in  conducting  a  successful  campaign  is  paramount.  From  before 
hostilities  begin,  through  the  end  of  the  campaign  and  beyond,  the  Joint  Force 
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Commander  (JFC)  must  continually  develop  and  refine  the  assessment  of  the  ever- 
changing  situation.  With  adequate  ISR  capability  the  JFC  has  the  means  to  provide  this 
assessment  and  in  certain  circumstances  to  attain  information  superiority.  Knowing  the 
enemy’s  vulnerabilities,  strengths,  and  intentions  allows  the  JFC  to  exploit  opportunities 
and  ultimately  defeat  the  enemy.  Intelligence  derived  from  good  ISR  also  provides  the 
ability  to  assess  the  effectiveness  of  operations  and  aids  in  determining  if  and  when  the 
overall  mission  has  been  accomplished.  Although  there  are  many  uses  and  purposes  for 
intelligence,  Joint  Pub  2-0,  “Joint  Doctrine  for  Intelligence  Support  to  Operations” 

[1 1 :3.3-6],  identifies  the  following  as  the  six  fundamental  intelligence  purposes: 

1 .  Supporting  the  Commander  -  Intelligence  should  provide  the  commander 
knowledge  of  the  situation  to  aid  in  optimal  decision-making  about 
operations. 

2.  Identifying  and  Determining  Objectives  -  Intelligence  should  allow 
commanders  to  determine  objectives  that  complement  national  security  policy 
objectives  and  the  derived  military  objectives. 

3.  Planning  and  Conducting  Operations  -  Intelligence  should  provide  necessary 
information  to  develop,  plan  and  execute  combat  operations. 

4.  Security  of  Operations/ Avoiding  Deception  and  Surprise  -  Intelligence 
systems  should  be  used  to  inhibit  the  enemy’s  attempt  at  deception  and 
surprise. 

5 .  Security  of  Operations  through  Deception  -  When  planning  operations,  the 
commander  should  have  knowledge  of  the  enemy’s  command  and  control 
systems  and  intelligence  systems  so  that  deception  can  be  used  against  the 
enemy. 

6.  Evaluating  the  Effects  of  Operations  and  Re-orienting  Forces  or  Terminating 
Operations  -  Intelligence  should  assist  in  evaluating  operational  results  and 
determining  if  objectives  have  been  met. 
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3.4  ISR  Principles 

Commanders  and  decision-makers  at  all  levels  of  war  depend  upon  ISR 
operations  to  reduce  uncertainties  and  to  make  better  decisions.  Certain  “principles”  or 
“attributes”  of  intelligence  are  identified  in  Air  Force  Doctrine  Document  2-5.2  (Draft), 
“Intelligence,  Surveillance,  and  Reconnaissance”  [15:1 1-15]  and  Joint  Pub  2-0  [1 1 :4.14- 
16]  which  increase  the  responsiveness  of  ISR  operations  to  commanders’  needs.  No  one 
principle  or  attribute  is  more  important  than  another  as  most  depend  upon  each  other,  and 
sometimes  trade-offs  must  be  made.  All  of  them  must  be  considered  to  maximize  ISR 
operational  effectiveness.  These  principles  are: 

1 .  Accuracy  -  Intelligence  reports  will  rarely  be  1 00  percent  accurate  because  of 
influences  such  as' human  error,  the  ‘fog  of  war’,  the  sophistication  of  enemy 
systems,  deception,  and  our  own  technical  limitations.  However,  ISR 
personnel  must  rely  on  their  knowledge  and  experience,  when  corroborating 
all  available  information  to  provide  the  most  accurate  picture  possible. 
Because  absolute  certainty  cannot  be  achieved,  a  confidence  level  of  the 
information  should  be  provided,  and  commanders  must  be  made  aware  of  the 
limitations  of  the  intelligence  estimate  given  to  them  [34:2].  A  confidence- 
level  scale  such  as  that  shown  in  Figure  5  can  be  used  to  communicate  the 
assessment  of  the  intelligence  provided. 

2.  Timeliness  -  To  be  used  effectively,  or  to  even  be  used  at  all,  intelligence 
must  be  received  before  a  decision  is  made  or  an  opportunity  is  missed.  To 
expend  resources  to  gather  intelligence,  but  then  not  have  the  intelligence 
available  in  time  to  be  used  defeats  the  purpose  of  ISR  operations.  New 
technology  has  influenced  this  principle  more  than  any  other,  and  continual 
improvements  are  being  made  due  to  its  utmost  importance  [34:3]. 

3.  Objectivity  -  Intelligence  must  be  “...unbiased,  undistorted,  and  free  from 
political  or  other  constraints”  [1 1 :4.14].  This  relates  to  the  principle  of 
accuracy.  Intelligence  must  not  be  manipulated  to  achieve  a  desired  result  or 
support  a  preconception. 
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Figure  5.  Confidence  Levels  of  Intelligence  Estimates  [1 1:4.13] 


4.  Unity  of  Effort/Interoperability  -  Cooperation  between  the  various  military 
services  and  nationalities  must  exist  at  all  levels  of  ISR  operations. 
Information  must  be  shared  to  minimize  duplication  and  maximize 
effectiveness.  Cross-cueing  and  analytical  exchange  can  significantly 
increase  operational  effectiveness.  Interoperability  among  systems  is  a  must 
for  this  to  occur. 

5.  Relevance  -  Intelligence  produced  should  be  applicable  to  determining, 
planning,  conducting,  and  evaluating  operations  [15:12].  It  should  enhance 
situational  awareness  and  support  current  and  future  operations.  Relevance 
must  be  considered  when  planning  for  collection  to  ensure  the  user’s 
requirements  are  met. 

6.  Usability  -  Intelligence  must  be  disseminated  in  a  form  that  does  not  require 
any  additional  processing  or  analysis.  The  commander,  planner,  or  war 
fighter  should  be  able  to  quickly  identify  and  apply  the  intelligence  received. 
Producers  and  users  of  intelligence  should  use  common  terminology  and 
common  means  of  communications  to  aid  in  usability .  Producers  must  also 
understand  the  circumstances  in  which  users  are  applying  the  intelligence 
received  in  order  to  best  meet  their  needs. 

7.  Completeness  -  Commanders  should  have  all  available,  relevant  information 
required  to  accomplish  the  mission.  Prioritization  of  requirements  aids  in 
ensuring  that  the  most  essential  information  is  provided  first. 
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8.  Readiness  -  Intelligence  organizations  need  to  stay  abreast  of  developing 
international  situations  and  correctly  identify  potential  trouble  spots.  During  a 
crisis,  the  ability  to  anticipate  possible  intelligence  needs  increases  the 
responsiveness  of  intelligence  to  the  commanders  that  need  it. 

9.  Fusion  -  Fusion  helps  to  overcome  the  inherent  weaknesses  of  individual 
collection  systems  and  the  deception  efforts  of  the  enemy  by  combining 
information  from  multiple  sources  and  analyzing  it  to  produce  a 
comprehensive  intelligence  product.  The  process  of  fusing  data  takes  time 
and  counteracts  the  principle  of  timeliness.  Although  a  fused  product  can 
increase  usability  and  accuracy,  a  balance  must  be  sought  with  timeliness. 
Fusion  is  not  always  required.  Often,  near-real-time  collections  must  be  acted 
upon  without  correlation  in  a  reactive  situation.  In  all  cases,  however, 
reliance  on  one  source  of  information  leaves  the  JFC  subject  to  deception, 
which  could  outweigh  the  advantage  of  timeliness. 

10.  Accessibility  -  Raw  data  must  be  accessible  to  ISR  operators  and  analysts  in 
order  for  the  ISR  process  to  continue  and  for  dissemination  to  take  place. 

Users  must  also  have  access  to  the  final  products.  A  push/pull  system  in 
which  users  can  “pull”  the  information  they  want  and  need  provides  them  with 
relevant  information  without  overloading  them  by  “pushing”  too  much 
information.  Security  must  be  maintained,  but  classifying  intelligence  at  the 
lowest  level  possibly  allows  for  greater  accessibility. 

1 1 .  Security  -  We  must  protect  our  own  capabilities  and  intentions,  and  safeguard 
our  sources  of  intelligence.  Declassification,  sanitization,  and  releasability 
issues  must  be  understood  and  adhered  to  by  all  personnel.  Although 
overclassification  is  not  desirable,  using  the  need-to-know  principle  and 
multilevel  secure  communications  will  help  prevent  putting  sources  at 
unnecessary  risk  and  will  still  enable  users  to  receive  needed  information. 

12.  Survivability  and  Sustainability  -  The  ISR  system  must  not  depend  totally  on 
one  type  of  intelligence,  one  means  of  intelligence  production,  or  one  means 
of  dissemination.  The  system  must  be  survivable  with  adequate  redundancy. 
There  should  be  no  one  critical  node  that,  if  destroyed,  cripples  the  whole 
system. 

3.5  Intelligence  Support 

ISR  supports  conflict  at  all  levels  of  war  -  strategic,  operational,  and  tactical 
(Figure  6).  At  the  strategic  level,  ISR  is  used  to  formulate  policy,  plans,  and  strategy  at 
the  national  and  theater  levels.  Strategic  intelligence  is  used  to  form  an  overall  picture 
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before  a  crisis  occurs,  and  then  to  enhance  the  overall  picture  once  a  crisis  begins.  It 
supports  national  objectives,  the  formulation  of  policies,  and  the  determination  of 
priorities.  Strategic  intelligence  is  also  used  to  determine  enemy  capabilities,  intentions, 
and  their  centers  of  gravity.  The  enemy’s  centers  of  gravity  are  those  “Characteristics, 
capabilities,  or  [locations]  from  which  a  military  force  derives  their  freedom  of  action, 
physical  strength,  or  will  to  fight”  [9:87],  such  as  the  mass  of  the  enemy  army,  the 
enemy’s  command  structure,  or  public  opinion. 

Operational-level  ISR  is  needed  to  aid  the  planning  and  conduct  of  major 
operations  within  the  theater.  It  focuses  on  intelligence  collection,  identification, 
location,  and  analysis  [19:6].  Identifying  the  enemy’s  capabilities  and  vulnerabilities  is 
essential  for  operational  activities  such  as  determining  targets  or  achieving  air  superiority. 

Tactical  intelligence  is  required  for  planning  and  conducting  tactical  operations  at 
the  component  or  unit  level  and  focuses  more  on  battles  and  engagements  than  long- 
range-planning  [19:6].  Intelligence  support  to  the  tactical  level  of  operations  “...primarily 
focus  on  support  to  mission  planning,  targeting  and  combat  assessment”  [15:11]. 

3.6  Types  of  Intelligence 

Imagery  Intelligence  (IMINT)  is  collected  using  “...visual  photography,  infrared 
sensors,  lasers,  electro-optics  and  radar  sensors. .  .where  images  of  objects  are  reproduced 
optically  on  film,  electronic  display  devices  or  other  media”  [34:16].  The  main  types  of 
images  produced  are  called  optical  and  non-optical  images.  “Optical  imagery  uses 
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Figure  6.  Intelligence  Support  at  all  Levels  of  War  [4] 

natural  illumination  in  the  portion  of  the  spectrum  that  humans  can  perceive  unaided. 
Non-optical  imagery  includes  infrared  and  radar”  [15:25].  Weather  effects,  such  as 
moisture  in  the  air  or  cloud  cover,  can  inhibit  various  sensor’s  ability  to  collect  IMINT. 

Signals  Intelligence  (SIGINT)  provides  information  based  on  enemy 
electromagnetic  transmissions.  SIGINT  is  categorized  into  Communications  Intelligence 
(COMINT),  and  Electronic  Intelligence  (ELINT).  ELINT  can  be  broken  down  further 
into  Foreign  Instrumentation  Signals  Intelligence  (FISINT),  Telemetry  Intelligence 
(TELINT),  and  Radar  Intelligence  (RADINT).  COMINT  involves  intercepting  and 
monitoring  enemy  communications.  Although  COMINT  is  an  excellent  source  for 
discovering  enemy  intentions,  among  its  drawbacks  are  “...susceptibility  to  deception,  the 
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requirement  for  linguists,  and  the  requirement  to  have  line  of  sight  with  the  transmitter 
for  very  high  frequency  and  ultra  high  frequency  systems”  [34:15].  ELINT  involves 
intercepting  and  monitoring  non-communication  emissions,  such  as  radar.  As  with 
COMINT,  it  also  “...is  susceptible  to  deception  and  suffers  from  a  line-of-sight 
requirement”  [34:16].  Within  ELINT,  FISINT  contains  the  “...technical  information 
derived  from  the  intercept  of  electromagnetic  emissions,  such  as  telemetry,  associated 
with  the  testing  and  operational  deployment  of  foreign  aerospace,  surface,  and  subsurface 
systems”  [15:26]. 

Measurement  and  Signature  Intelligence  (MASINT)  is  defined  as  “The 
information  derived  from  the  exploitation  of  sensor  measurement  and 
signatures... collected  by  ground,  airborne,  and  space  systems,  to  identify  distinctive 
features  of  the  source,  emitter,  or  sender,  such  as  infrared  signatures  or  unique  sound 
characteristics”  [1 5 :26].  MASINT  includes  a  variety  of  different  types.  (See  Table  2). 

A  disadvantage  of  MASINT  is  the  extended  amount  of  time  needed  to  produce  the 
MASINT  product. 

Human  Intelligence  (HUMINT)  is  the  collection  of  information  by  people  with 
first-hand  knowledge  of  an  enemy  situation.  HUMINT  can  result  from  many  different 
circumstances.  Examples  of  HUMINT  collection  may  be  information  collected  by 
Special  Forces  missions,  aircrew  debriefings,  or  enemy  prisoners  of  war. 

Open  Source  Intelligence  (OSINT)  is  information  collected  through  media 
available  to  the  general  public.  Foreign  newspapers,  radio,  television,  and  the  internet 
can  provide  valuable  information  on  enemy  knowledge  and  consciousness. 
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Technical  Intelligence  (TECHINT)  is  derived  from  the  exploitation  of  foreign 
material.  “Technical  intelligence  begins  when  an  individual  service  member  finds 
something  new  on  the  battlefield  and  takes  the  proper  steps  to  report  it.  The  item  is  then 
exploited  at  succeedingly  higher  levels  until  a  countermeasure  is  produced  to  neutralize 
the  adversary’s  technological  advantage”  [11:GL13], 

Counterintelligence  (Cl)  is  information  gathered  to  protect  against  espionage, 
sabotage,  assassinations,  or  terrorism  conducted  by  or  on  behalf  of  foreign  governments 
or  foreign  persons. 


Table  2.  Types  of  Intelligence  [1 1:2.2] 


IMINT 

Imagery  Intelligence 

PHOTOINT 

Photo  Intelligence 

SIGINT 

Signals  Intelligence 

COMINT 

Communications  Intelligence 

ELINT 

Electronic  Intelligence 

FISINT 

Foreign  Instrumentation  Signals  Intelligence 

TELINT 

Telemetry  Intelligence 

RADINT 

Radar  Intelligence 

HUMINT 

Human  Intelligence 

MASINT 

Measurement  and  Signature  Intelligence 

ACINT 

Acoustical  Intelligence 

OPINT 

Optical  Intelligence 

ELECTRO- 

Electro-optical  Intelligence 

IRINT 

Infrared  Intelligence 

LASINT 

Laser  Intelligence 

NUCINT 

Nuclear  Intelligence 

RINT 

Unintentional  Radiation  Intelligence 

OSINT 

Open  Source  Intelligence 

TECHINT 

Technical  Intelligence 

Cl 

Counterintelligence 
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3.7  Intelligence  Community 

Many  agencies  make  up  what  is  known  as  the  “Intelligence  Community”.  These 
agencies  are  comprised  of  both  military  and  civilian  personnel  and  are  shown  in  Figure  7 
Each  element  of  the  community  has  its  own  defined  function,  but  the  interaction  and  the 
cooperation  among  the  individual  elements  is  needed  to  support  the  ISR  mission. 


Figure  7.  Intelligence  Community  Membership  [13:2.3] 


3.8  Communications  for  ISR  Operations 

Communication  is  a  cornerstone  of  any  military  operation.  The  ISR  process  and 
its  success  also  rest  on  the  ability  to  communicate.  Requesting  information,  tasking  ISR 
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assets,  collecting  data,  and  disseminating  information,  all  rely  on  communication  systems 
to  fulfill  their  purpose.  There  are  a  myriad  of  Command,  Control,  Communications, 
Computer,  and  Intelligence  (C4I)  systems  in  existence,  and  most  of  them  depend  upon 
satellites  to  relay  necessary  communications  such  as  voice,  fax,  message,  raw  data, 
imagery,  and  video.  Communication  satellites  allow  intelligence  personnel  to  analyze 
data  from  a  location  hundreds  or  thousands  of  miles  from  the  battlefield. 

The  military  depends  upon  its  own  communication  satellites  (MILSATCOM),  as 
well  as  commercially-leased  communication  satellites  to  move  the  massive  amounts  of 
information  needed  during  both  peacetime  and  conflict.  In  addition  to  the  International 
Maritime  Satellite  System  (INMARSAT)  and  the  International  Telecommunications 
Satellite  (INTELSAT),  which  are  the  primary  international  commercial  satellite  carriers 
leased  by  the  DoD  [14:7.1 1-14],  there  are  numerous  domestic  companies  which  provide 
leased  communications  through  geosynchronous  satellites.  A  handful  of  these 
communication  satellites  are  summarized  below. 

3.8.1  Communication  Satellites 

3.8.1.1  Defense  Satellite  Communications  System  (DSCS) 

The  DSCS-III  constellation  currently  has  nine  multi-channel  Super-High 
Frequency  (SHF)  satellites  in  geosynchronous  orbits.  DSCS-III  can  selectively  negate 
jamming  directed  at  it  with  little  impact  to  its  transmission  capabilities.  It  also  contains  a 
“. .  .Ultra-High  Frequency  (UHF)  single  channel  transponder  package  that  serves 
AFSATCOM  users  providing  increased  capabilities  in  a  stressed  environment”  [14:7.23]. 
“DSCS  provides  the  backbone  for  the  transmission  of  high  capacity  command  and 
control,  intelligence  and  multi-channel  communications  service”  [10:33].  Additionally 
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DSCS  directly  supports  the  Global  Command  and  Control  System  (GCCS).  “DSCS  earth 
stations  connect  to  major  voice,  data,  and  message  switching  centers  that  rapidly  link 
critical  circuits  and  systems  to  the  Defense  Information  System  Network  (DISN)  and 
commercial  networks”  [10:33]. 

3.8.1.2  Milstar 

Milstar  is  the  next  generation  military  satellite  communication  system  designed  to 
provide  survivable,  jam-resistant  command  and  control  communications  for  strategic  and 
tactical  forces  worldwide.  The  Milstar  constellation  will  consist  of  four  satellites  in  low 
inclined  near-geosynchronous  orbit  operating  in  the  Extremely-High  Frequency  (EHF) 
band.  “An  advantage  of  Milstar  over  DSCS  and  UHF  satellites  is  that  it  has  a  cross- 
linking  capability  that  will  allow  direct  transmission  of  communications  from  one 
satellite  to  another  without  the  intervention  of  ground  relay  stations”  [14:7.46]. 

3.8.1.3  NATO  Satellite  System 

The  NATO  IV  system  is  a  single  satellite  positioned  over  the  Atlantic  Ocean, 
along  with  27  satellite  ground  terminals,  2  control  centers,  and  the  NATO  school  segment 
at  Latina,  Italy  [37].  The  NATO  IV  satellite  has  four  SHF  channel  and  two  UHF 
channels  used  “...primarily  for  diplomatic  and  military  communications  and  intelligence 
support  to  the  Commander-in-Chief  of  NATO  and  to  the  National  Command  Authority  of 
NATO  forces”  [14:7.47].  The  SHF  footprint  for  the  NATO  IV-A  contains  Eastern 
Canada,  the  Atlantic,  parts  of  North  Africa,  Europe,  and  Southeastern  Greenland.  The 
UHF  footprint  contains  the  Eastern  United  States,  the  Atlantic,  South  America,  Africa, 
and  most  of  Greenland  [37]. 
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3.8.1.4  Tracking  and  Data  Relay  Satellite  System  (TDRSS) 

The  TDRSS  constellation  consists  of  a  small  number  of  satellites  in 
geosynchronous  orbit  that  support  near-real-time  data  transmission  from  low-earth 
orbiting  reconnaissance  satellites  over  approximately  85  percent  of  the  earth’s  surface. 
“TDRSS  offers  a  useful  contingency  capability,  and  the  military  pays  $100M  annually  to 
use  TDRSS”  [14:7.48-49]. 

3.8.1.5  UHF  Follow-On  (UHF  F/O)  Program 

The  US  Navy  is  deploying  a  new  constellation  of  UHF  satellites  to  replace  their 
current  Fleet  Satellite  Communications  network;  these  satellites  are  called  UHF  F/O,  or 
UFO  satellites.  The  plan  is  to  have  eight  satellites  in  orbit,  with  two  on-orbit  spares.  The 
first  UFO  satellite  was  launched  in  1994.  These  new  satellites  will  “...employ  Demand 
Assigned  Multiple  Access  (DAMA)  time  division  multiplexing  techniques  to  allot 
capacity  to  more  users.  Demand-based  assignment  means  that  unused  transponder  space 
is  dynamically  reallocated  in  real-time  based  on  precedence,  greatly  improving 
information  throughput”  [14:7.53].  The  last  six  satellites  to  deployed  will  also  carry  EHF 
packages  to  improve  “...anti-jam  telemetry,  command,  broadcast  and  fleet 
communications.... The  EHF  modes,  formats,  and  data  rates  will  be  subsets  of  those 
employed  on  the  MILSTAR  system.  Satellites  8, 9, 10  will  also  provide  Global 
Broadcast  (Joint  Broadcast)  Service  (GBS)”  [14:7.53]. 

3.8.1.6  International  Maritime  S  atellite  System  (INMARSAT) 

The  International  Maritime  Satellite  Organization  is  headquartered  in  London, 
England  with  79  member  nations.  INMARSAT  provides  voice,  message,  facsimile  and 
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data  communications  through  four  satellites  in  geosynchronous  orbit  and  leased 
transponders  on  other  communications  satellites  [14:7.12-13]. 

3.8.1.7  International  Telecommunications  Satellite  (INTELSAT) 

INTELSAT  provides  telephone,  television  and  data  distribution  services  to  people 
on  every  continent,  and  is  the  world’s  largest  commercial  satellite  communications 
provider.  [14:7.14]. 

3.8.2  C4  Architectures 

The  military  employs  a  variety  of  Command,  Control,  Communications,  and 
Computers  (C4)  architectures  as  well.  A  few  examples  of  these  relating  to  ISR  include 
Defense  Information  Systems  Network  (DISN),  Global  Command  and  Control  System 
(GCCS),  and  Global  Broadcast  Service  (GBS).  These  systems  are  briefly  explained 
below. 

3.8.2.1  Defense  Information  System  Network  (DISN) 

“DISN  provides  Defense-wide  communications  for  the  day-to-day  operations  of 
the  DoD  and  services  at  the  core  of  DoD  wartime  communications  for  the  National 
Command  Authority  (NCA),  the  Joint  Chiefs  of  Staff,  and  Commanders-in-Chief 
(CINC),  and  other  critical  users”  [10:25].  It  is  the  DoD’s  worldwide  telecommunications 
network  that  also  incorporates  interoperability  with  allied  and  coalition  forces  [42]. 

3.8.2.2  Global  Command  and  Control  System  (GCCS  ) 

The  Global  Command  and  Control  System  (GCCS),  which  replaced  the  World 
Wide  Military  Command  and  Control  System,  is  a  comprehensive  worldwide  system 
providing  the  National  Command  Authority  (NCA),  Chairman  of  the  Joint  Chiefs  of 
Staff,  combatant  commanders,  Services,  Defense  Agencies,  Joint  Task  Force 
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commanders  and  component  commanders  information  processing  and  dissemination 
capabilities  to  conduct  Command  and  Control  operations.  GCCS  has  numerous  functions 
including  providing  situational  awareness,  readiness  assessments,  course  of  action 
development,  imagery  exploitation,  intelligence  mission  supports,  crisis  planning, 
deliberate  planning,  operational  plan  generation,  deployment  of  forces,  indications  and 
warning,  and  real-time  combat  execution  from  a  C4I  perspective  [10:47]. 

3.8.2.3  Global  Broadcast  Service  (GBS) 

The  Global  Broadcast  Service  system,  in  development,  will  be  a  space-based, 
high  data  rate  communications  link  for  a  one-way  flow  of  intelligence,  weather,  and  other 
information  over  a  widespread  geographic  area. 

The  GBS  will  be  a  system  of  uplink  sites,  broadcast  satellites,  receiver  terminals, 
and  management  processes  for  requesting  and  coordinating  the  distribution  of 
information  products.  Each  GBS  satellite  in  a  near-worldwide  constellation  will  be 
served  by  a  primary  uplink  site  where  information  products  are  assembled  and 
transmitted  to  a  high-powered  satellite  for  relay  to  forces  over  a  large  area.  GBS  will 
also  have  the  capability  to  inject  products  directly  from  the  theater  it  serves.  A  big 
advantage  of  GBS  is  that  with  small  receive  terminals,  mobile  forces  are  no  longer 
restricted  by  the  requirement  for  large,  fixed  antennas  to  receive  information  formerly 
available  only  to  command  centers  [10:43]. 

“GBS  is  an  extension  of  the  Defense  Information  Systems  Network  (DISN)  and  a 
part  of  the  overall  DoD  MILSATCOM  Architecture.  It  will  interface  with,  and  augment 
other  major  DoD  information  systems,  such  as  the  Global  Command  and  Control  System 
(GCCS),  as  well  as  other  theater  information  management  systems”  [21]. 
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4.  Intelligence.  Surveillance.  Reconnaissance  Process  during  Theater-level  Conflict 


The  Intelligence,  Surveillance,  Reconnaissance  (ISR)  process  must  be  coordinated 
at  all  levels  of  command  and  among  all  branches  of  the  military,  as  well  as  with  national 
agencies.  ISR  involves  many  diverse  assets  that  synergistically  provide  the  fullest 
understanding  of  the  enemy,  his  capabilities,  and  his  weaknesses.  “This  understanding 
directly  supports  formulating  military  objectives  and  strategy,  determining  planning  and 
conducting  military  operations,  and  identifying  the  adversary’s  strategic,  operational, 
tactical  centers  of  gravity”  [15:3].  This  chapter  presents  the  ISR  process  used  to 
accomplish  the  purposes  of  ISR  presented  in  Chapter  3.  The  overall  process,  with  some 
lower-level  details,  is  presented  to  demonstrate  the  vast  amount  of  operations, 
coordination,  and  diversity  of  elements  that  must  all  function  together  for  the  process  to 
be  successful. 

4.1  Process  Overview 

The  ISR  process  encompasses  the  methodology  of  transforming  a  need  for 
information  into  a  product  that  fulfills  that  need.  The  ISR  process  entails  seven  main 
steps:  Plan,  Task,  Collect,  Analyze,  Disseminate,  Evaluate,  and  Apply.  This  process 
cycles  throughout  a  conflict  and,  in  certain  cases,  is  modified  and  adapted  to  the  scenario 
at  hand.  For  example,  in  some  cases,  analysis  of  raw  data  is  not  necessary,  or  even 
desirable,  and  therefore  that  step  in  the  ISR  process  may  be  bypassed.  Also,  it  should  be 
realized  that  data  is  evaluated  throughout  the  process  to  ensure  sensors  are  functioning 
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properly,  and  that  the  intended  information  is  available.  The  ISR  process  is  depicted  in 
Figure  8.  Each  step  in  the  process  is  explained  below. 


I 


Requirement  Satisfied 

Figure  8.  Intelligence,  Surveillance,  and  Reconnaissance  Process  [15:24] 

4.1.1  User  Requirement 

During  the  transition  from  peace  to  hostilities,  the  NCA  tasks  the  appropriate 
CINC  in  the  theater  where  military  operations  will  occur.  The  CINC  can  assume  the 
responsibility  of  JFC  or  delegate  the  authority  to  a  subordinate.  “The  JFC  is  the 
commander  of  a  unified  command,  subunified  command,  or  Joint  Task  Force  (JTF) 
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authorized  to  exercise  operational  authority  over  a  joint  force  to  accomplish  an  assigned 
mission.  The  JFC  determines  appropriate  military  objectives  and  sets  priorities  for  the 
entire  joint  force”  [17:54]. 

Subordinate  to  the  Joint  Force  Commander  are  the  Commanders  for  the  Joint 
Force  Air  Component  (JFACC),  the  Joint  Force  Land  Component  (JFLCC),  the  Joint 
Force  Maritime  Component  (JFMCC),  and  the  Joint  Force  Special  Ops  Component 
(JFSOCC).  Each  of  these  Component  Commanders  translates  JFC  objectives  into 
military  operations.  To  conduct  military  operations,  Component  Commanders  require 
knowledge  of  the  enemy  and  of  the  effectiveness  of  completed  missions. 

Although  intelligence  requirements  emanate  from  any  level  of  command,  the 
commander’s  information  requirements  are  the  principal  drivers  of  the  intelligence  effort. 
For  this  study,  the  JFC  is  considered  the  primary  user,  the  one  setting  requirements  for 
the  overall  process. 

The  JFACC  is  the  primary  player  in  requesting  intelligence  support  in  this  ISR 
process.  The  JFACC  has  the  responsibility  of  overseeing  all  air  and  space  operations, 
which  form  the  backbone  of  intelligence  operations.  The  JFACC  “. .  .recommends  the 
proper  employment  of  air  and  space  forces.... The  JFACC  also  plans,  coordinates, 
allocates,  tasks,  executes,  and  assesses  air  and  space  operations  to  accomplish  assigned 
operational  missions”  [17:59].  The  JFACC  is  responsible  for  the  production  and 
execution  of  the  daily  ATO.  A  key  ingredient  in  the  production  of  an  effective  ATO  is 
having  timely  and  accurate  intelligence. 

Requirements  for  intelligence  tie  to  past,  present,  and  future  operations.  For 
example,  the  JFACC  must  know  the  effectiveness  of  air  strikes  to  make  re-attack 


35 


decisions;  surveillance  helps  in  acquiring  targets;  and  knowledge  of  enemy  air  defense 
sites  dictates  air  strike  ingress  and  egress  routes. 

All  levels  of  war  have  slightly  differing  requirements  and  uses  of  intelligence.  At 
the  strategic  level,  campaign  planners  may  need  information  about  enemy  intentions  and 
capabilities,  available  resources,  and  the  geography  of  the  battlefield.  At  the  operational 
level,  enemy  doctrine,  personalities  of  enemy  commanders,  and  enemy  centers  of  gravity 
are  of  interest.  At  the  tactical  level,  the  number,  types,  mobility,  and  equipment  of  enemy 
forces  are  required.  Commanders  require  quantitative  items  such  as  the  status  of  a  bridge 
and  the  number  of  tanks;  however,  the  commander  also  needs  information  on  the 
intangibles,  such  as  how  the  enemy  views  his  potential  courses  of  action,  and  which  are 
the  most  likely  to  follow. 

4.1.2  Planning 

Intelligence  organizations  at  the  national  level,  in  the  combatant  commands,  and 
subordinate  joint  forces  all  interact  and  support  each  other  in  order  to  fill  JFC 
requirements.  The  National  Military  Joint  Intelligence  Center  (NMJIC),  the  combatant 
command  intelligence  officer  (J-2),  the  Joint  Intelligence  Center  (JIC),  the  subordinate 
joint  force  J-2  and  Joint  Intelligence  Support  Element  (JISE)  are  all  responsible  for 
providing  intelligence  support  to  military  operations. 

The  NMJIC  is  the  national  agency  that  serves  as  the  focal  point  for  all  defense 
intelligence  activities  in  support  of  joint  operations.  It  also  provides  a  conduit  to  the 
entire  DoD  intelligence  community  and  organization  in  support  of  joint  operations 
[ll:xi]. 
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At  the  theater  level,  the  JIC  has  the  primary  responsibility  for  ensuring  that 
combatant  commanders  and  theater  forces  receive  the  required  intelligence  support 
[28:xi]. 

At  the  joint  task  force  level,  the  USE  is  responsible  for  the  collection,  analysis, 
and  fusion  of  intelligence,  as  well  as  the  dissemination  of  intelligence  and  intelligence 
products  for  the  joint  operations  area  [1 1  :xi]. 

Requirements  must  be  validated  by  the  combatant  command  J-2  with  the  JIC 
tracking  .  .the  status  of  research,  validation,  submission  and  satisfaction  of  all  collection 
requests  received...”  [12:3.15]. 

After  a  requirement  has  been  validated,  it  is  determined  whether  a  new  collection 
effort  is  needed  or  if  the  information  has  already  been  collected.  If  a  new  collection  is 
needed,  planning  begins  to  fulfill  the  requirement.  Planning  for  ISR  collection  entails 
prioritizing  requests;  determining  available  assets,  their  capabilities,  as  well  as  their 
vulnerabilities;  and  considering  the  time  constraints  of  the  request  versus  the  timeliness 
of  an  asset’s  response  [15:17].  Planning  for  intelligence  collection  is  usually  done  in 
conjunction  with  operational  planning.  Intelligence  planning  also  requires  coordinating 
priorities  with  national-level  and  other  theater-level  agencies,  and  ensuring 
communications  support,  manpower  and  equipment  are  available. 

The  Joint  Air  Operations  Center  (JAOC)  supports  the  JFACC  in  planning  theater 
ISR  missions.  It  “...is  the  air  and  space  operations  planning  and  execution  focal  point  for 
the  JTF  and  is  where  centralized  planning,  direction,  control,  and  coordination  of  air  and 
space  operations  occurs”  [17:75].  Since  intelligence  collection  is  closely  tied  to 
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operations,  the  JAOC  must  work  intimately  with  the  JIC  in  planning  for  air  and  space 
intelligence  collection. 

The  Joint  Staff  J-2  Deputy  Directorate  for  Targeting  Support,  J-2T,  is  the  manager 
for  target  intelligence  from  national  systems.  The  J-2T  operates  the  NMJIC  Targeting 
and  Battle  Damage  Assessment  (BDA)  Cell,  which  is  the  single  national-level  source  of 
targeting  and  BDA  support  to  the  JCS  and  combatant  commands  [13:5.8]. 

Planning  for  intelligence  collection  involves  deciding  which  sensor  or  system  can 
best  fill  the  requirement.  This  decision  depends  on  numerous  factors.  Sensor  capabilities 
are  considered  in  conjunction  with  the  target  characteristics  and  location  and  the  type  of 
intelligence  required.  An  asset’s  coverage  footprint  and  its  time  over  a  target  must  meet 
collection  requirements.  Timeliness  is  an  important  factor  when  deciding  which  sensor 
to  use;  information  that  arrives  too  late  is  of  no  use.  Planners  need  to  think  about  the 
availability  of  resources,  the  time  needed  to  plan  the  mission,  collect  the  data,  process 
and  analyze  the  data  and  disseminate  the  results.  Environmental  factors  including  the 
threat,  terrain,  and  weather  need  to  be  examined.  Threats  to  assets  must  be  identified.  A 
sensor  that  must  fly  over  a  target  is  more  vulnerable  than  a  standoff  system;  satellite 
sensors  are  the  least  vulnerable  assets.  Terrain  may  interfere  with  sensor  capabilities  that 
require  line-of-sight.  Weather  conditions  such  as  cloud  cover  or  rain  can  inhibit  certain 
sensors’  capabilities.  Each  of  these  factors  must  be  considered,  and  weighed  against  each 
other,  when  planning  for  collection.  Figure  9  shows  the  collection  planning  factors  that 
need  to  be  considered  in  sensor  or  system  selection. 
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Resource  Considerations 


Target  Situation 


Figure  9.  Collection  Planning  Factors  [12:3.17] 

Consideration  is  also  given  to  how  many  sensors  are  needed.  Redundancy  may 
guarantee  the  information  is  obtained,  or  possibly  guarantee  more  accurate  information, 
but  it  will  be  at  the  cost  of  tying  up  limited  ISR  assets.  Collection  may  need  to  be 
coordinated  so  that  cross-cueing  between  sensors  can  be  used. 

Intelligence  requirements  are  filled  by  a  multitude  of  sensors,  platforms, 
communication  systems,  dissemination  systems,  and  analysts.  Not  all  products  are  used 
in  the  same  way  and  for  the  same  purpose.  Intelligence  products  can  be  categorized  into 
many  different  uses.  Some  of  these  categories  are  indications  and  warning,  near-real- 
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time  and  real-time  situational  awareness,  intelligence  preparation  of  the  battlespace, 
target  intelligence,  and  battle  damage  assessment.  Planning  for  ISR  activities  requires 
knowledge  of  the  product  required  as  that  will  have  a  big  impact  on  the  collector, 
platform,  and  dissemination  system  chosen. 

4.1.3  Tasking 

Tasking  converts  intelligence  requirements  into  collection  requirements.  The 
Combat  Plans  Division  in  the  JAOC  schedules  reconnaissance  missions  for  theater  ISR 
assets.  The  assets  are  then  tasked  through  the  JAOC’s  dissemination  of  an  ATO  directly 
to  the  units.  National  assets  and  HUMINT  collectors  are  tasked  slightly  differently.  For 
these  collection  requirements,  the  JIC  submits  the  request  to  the  NMJIC  for  tasking  of  the 
national  systems  and  HUMINT  collectors.  The  NMJIC  is  the  focal  point  for  intelligence 
activities  at  the  national  level  during  crises,  and  is  co-located  in  the  Pentagon  with  the 
National  Military  Command  Center.  A  National  Intelligence  Support  Team  (NIST) 
traditionally  deploys  to  the  JTF  headquarters  to  provide  an  additional  conduit  to  the 
NMJIC. 

National  intelligence  organizations  that  support  the  JFC  on  a  full-time  basis 
include:  Central  Intelligence  Agency  (CIA),  Defense  Intelligence  Agency  (DIA), 
National  Security  Agency  (NSA),  National  Imagery  and  Mapping  Agency  (NIMA),  and 
the  State  Department  [13:1.3].  Most  of  these  national  agencies  have  liaison  personnel  or 
support  teams  that  are  either  stationed  in  theater  or  deploy  to  the  theater  to  ease 
communication  and  improve  efficiency. 
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The  CIA  has  primary  expertise  in  HUMINT  collection,  imagery,  political  and 
economic  intelligence.  The  Office  of  Military  Affairs  within  the  CIA  is  the  point  of 
contact  for  the  CIA’s  military  support. 

The  DIA’s  mission  ranges  throughout  the  ISR  process,  from  planning  and 
directing  intelligence  to  carrying  out  BDA.  The  DIA’s  Directorate  for  Intelligence 
Operations  (DO)  oversees  collection  requirements  and  operations.  “The  DO  also  directs 
all  non-tactical  DoD  HUMINT  activities  through  the  Defense  HUMINT  Services  (DHS), 
and  measurement  and  signature  intelligence  (MASINT)  activities  through  the  Central 
MASINT  Office.  The  DHS  provides  HUMINT  resources  to  support  the  joint  force 
requirements”  [13:6.5]. 

The  NSA  “...provides  SIGINT  and  information  security  (INFOSEC), 
encompassing  communications  security  (COMSEC)  and  computer  security  as  well  as 
telecommunications  support  and  operations  security  (OPSEC)”  [13:7.1]. 

The  National  Imagery  and  Mapping  Agency’s  (NIMA)  primary  mission  is  to 
provide  imagery  intelligence  and  geospatial  information.  The  NIMA  coordinates 
imagery  collection;  manages  and  tasks  national  assets;  provides  advisory  tasking  for 
theater  assets;  and  processes,  exploits,  disseminates,  and  evaluates  imagery  and  IMINT. 

The  DIA  is  the  overall  coordinator  for  National  Reconnaissance  Office  (NRO) 
support  for  DoD.  The  NRO  provides  the  nation’s  space-based  reconnaissance 
capabilities  through  the  operation  of  IMINT  and  SIGINT  satellites.  “IMINT 
requirements  are  tasked  through  the  NIMA  and  SIGINT  requirements  through  the  NSA” 
[13:9.2]. 
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Each  military  service  has  intelligence  activities  supporting  their  individual 
missions.  Examples  of  some  of  these  organizations  are  US  Army  Intelligence  and 
Security  Command  (INSCOM),  National  Maritime  Intelligence  Center  (NMIC),  National 
Air  Intelligence  Center  (NAIC),  and  the  US  Marine  Corps  Intelligence  Activity  (MCIA). 

Communicating  with  the  appropriate  agency  for  tasking  an  asset  is  an  important 
endeavor  that  must  be  understood.  However,  even  after  the  appropriate  sensor/platform 
has  been  tasked,  it  is  very  possible  that  new,  and  possibly  higher  priority,  or  more  time- 
sensitive  collections  must  be  executed.  Some  missions  may  be  dynamically  re-tasked 
with  little  consequence,  while  re-tasking  may  be  impossible  for  others.  The  ability  to 
dynamically  re-task  assets  depends  on  the  situation,  the  time-sensitivity,  the  assets  and 
their  capabilities,  and  the  trade-offs  between  the  current  tasking  and  the  new  tasking. 

4.1.4  Collection 

Collection  operations  acquire  information  and  provide  it  to  processors  and 
disseminators.  Many  of  the  collection  planning  factors  are  considered  at  the  time  of 
collection.  Weather,  vulnerability,  threat,  capabilities,  along  with  other  risks  and  the 
overall  operational  situation,  all  determine  how  and  when  a  sensor  will  make  an  assigned 
collection.  “For  aerial-based  systems,  the  wing  or  squadron  commander  normally  has  the 
responsibility  to  accomplish  the  ISR  mission.  For  ground-based  systems,  like  Special 
Operations  Forces  (SOF)  or  HUMINT,  the  responsibility  lies  with  the  competent 
authority  at  the  tactical  level.  The  JFACC  is  the  final  authority  on  determining  whether 
the  benefits  of  successfully  accomplishing  the  mission  outweigh  the  risks  involved” 
[15:19]. 
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Many  different  sensors  are  involved  in  ISR  collection.  There  are  manned  and 
unmanned  systems;  airborne,  space,  and  ground  systems;  military,  non-military,  and 
national  systems;  and  technical  versus  human  collection.  Each  system  has  particular 
advantages,  disadvantages,  capabilities,  and  limitations.  One  may  be  accurate,  but  slow 
to  provide  information,  another  may  be  highly  survivable,  but  also  highly  predictable  and 
therefore  more  easily  deceived.  “It  is  desirable  to  have  sensors  cross-cue  each  other  to 
provide  synergism  that  capitalizes  on  individual  sensor  strength.  For  example,  J-STARS 
was  able  to  cross-cue  unmanned  aerial  vehicles  operated  by  the  Marines  during  Desert 
Storm.  This  allowed  the  Marines  to  pinpoint  Iraqi  defenses  and  monitor  troop 
movements...”  [34:50].  The  military  uses  a  mix  of  systems  to  accomplish  ISR 
operations.  Some  of  the  common  systems  in  today’s  ISR  environment  are  identified 
below. 

4.1.4.1  Airborne  systems 

4.1.4.1.1  Uninhabited  Aerial  Vehicles  (UAVs) 


Figure  10.  Predator  Medium  Altitude  Endurance  (MAE)  UAV 

The  greatest  advantage  of  UAVs  is  reduced  risk  to  friendly  personnel.  UAVs 
have  other  advantages  such  as  long  loiter  times  and  relative  cost  effectiveness  when 
compared  with  manned  aircraft  missions.  UAV  support  can  provide  near-real-time 
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intelligence  and  BDA  to  all  military  services.  Various  types  of  UAVs  provide  a  broad 
range  of  capabilities  and  uses.  Tactical  UAVs  include  Pioneer  and  Hunter,  while  UAVs 
with  more  of  a  focus  on  longer  ranges  and  longer  dwell  times  include  Predator,  Global 
Hawk,  and  DarkStar  [49].  The  main  sensors  found  on  UAVs  are  for  SIGINT,  Synthetic 
Aperture  Radar  (SAR),  Electro-optical  imagery,  multispectral  imagery,  and  real-time 
video  imagery  [15:27].  The  UAV  guidance  and  control  systems  are  either  remotely 
controlled,  preprogrammed,  or  some  combination  of  both. 

4.1.4.1.2  RC-135 


Figure  11.  RC-135V/W  RIVET  JOINT 

The  RC-135  is  an  Air  Force  reconnaissance  version  of  a  Boeing  707.  The  RC- 
135’ s  are  considered  national  assets,  but  may  be  tasked  by  the  theater  commander  during 
wartime  for  tactical  roles  [34:16].  Various  models  of  the  RC-135  such  as  RC-135  U 
Combat  Sent,  RC-135  V/W  Rivet  Joint,  RC-135  X  Cobra  Eye,  and  RC-135  Cobra  Ball 
all  provide  SIGINT,  to  include  COMINT  and  ELINT,  with  Cobra  Eye  and  Cobra  Ball 
also  providing  MASINT  [27]. 
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4.1.4.1.3  E-8C  J-STARS 


Figure  12.  E-8C  J-STARS 


The  E-8C  Joint  Surveillance  Target  Attack  Radar  System  (J-STARS)  is  a 
modified  Boeing  707  with  a  primary  mission  of  air-to-ground  surveillance.  An  Army  and 
Air  Force  multi-service  system,  J-STARS  is  used  to  locate,  identify,  and  track  ground 
targets  -  both  fixed  and  mobile  -  in  near-real-time.  It  primarily  focuses  on  non-emitting 
targets  making  radar  imagery  intelligence  its  principal  product  [34:18].  J-STARS  has  a 
long  loiter  time  and  a  long  range  and  can  look  deep  into  enemy  territory  to  detect  and 
track  ground  movement  in  both  forward  and  rear  areas.  It  reports  enemy  location,  size, 
direction,  rate  of  movement,  and  type  of  target  [10]. 

J-STARS  uses  a  phased  array  radar  for  collection  with  a  range  of  over  150  miles  [30]. 
Comprehensive  communications  allow  J-STARS  to  provide  voice  and  data  transmissions 
to  a  variety  of  platforms,  weapons  system,  and  ground  modules.  Satellite 
communications  allow  J-STARS  to  pass  information  to  users  not  in  its  line-of-sight. 
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The  U-2  is  a  high-altitude  reconnaissance  aircraft  capable  of  gathering  multi- 
sensor  photo,  electro-optic,  infrared  and  radar  imagery,  as  well  as  electronic  intelligence 
[48].  The  U-2  has  proven  to  be  a  very  reliable  aircraft  with  high  mission  completion 
rates.  Although  it  is  not  air-refuelable,  it  can  fly  for  more  than  twelve  hours,  if  necessary. 
The  U-2  has  a  long  standoff  range  and  can  down-link  saved  or  near-real-time  imagery  to 
ground  stations  via  line-of-sight  transmissions  [46]. 

4.1.4.1.5  EH-60L  Quick  Fix 


Figure  14.  EH-60L  Quickfix 


The  EH-60L  is  an  Army  tactical  helicopter  that  provides  SIGINT  and  electronic 
countermeasures  (ECM)  support  to  Army  units.  The  data  downlinked  in  near-real-time 


provides  targeting  information.  System  information  provides  input  for  the  tasking  and 
mission  direction  of  other  assets  [40]. 

4.1.4.1.6  E-3  Sentry  Airborne  Wa  rning  &  Control  System  (AW ACS) 


Figure  15.  E-3  Sentry 


The  E-3  Sentry,  AWACS,  is  a  modified  Boeing  707/320  commercial  airframe 
with  a  rotating  radar  dome.  It  provides  all-weather  surveillance,  command,  control  and 
communications  to  U.S.  and  NATO  forces  and  has  proven  to  be  the  “...premier  air  battle 
command  and  control  aircraft  in  the  world  today”  [20].  The  radar  has  a  range  of  more 
than  200  miles  for  low-flying  targets  and  a  significantly  greater  range  for  aircraft  flying  at 
medium  or  high  altitudes.  The  E-3  also  has  navigation,  communications,  and  data 
processing  capabilities.  Console  operators  view  data  in  graphic  or  tabular  formats  to 
perform  “. .  .surveillance,  identification,  weapons  control,  battle  management  and 
communications  functions”  [20]. 

The  E-3  systems  gather  detailed  battlefield  information  such  as  position  and 
tracking  of  enemy  aircraft  and  ships,  and  the  location  and  status  of  friendly  aircraft  and 
ships.  “The  information  can  be  sent  to  major  command  and  control  centers  in  rear  areas 
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or  aboard  ships.  In  time  of  crisis,  this  data  can  be  forwarded  to  the  National  Command 
Authority  in  the  United  States”  [20]. 


The  E-3  supports  air-to-ground  operations  by  providing  information  for 
interdiction,  reconnaissance,  airlift,  and  close-air  support.  For  air  defense,  the  E-3  can 
detect,  identify,  and  track  enemy  forces  and  direct  fighter-interceptor  aircraft  to  enemy 
targets  [32]. 

4.1.4.1.7  RC-12  Guardrail  Common  Sensor 


Figure  16.  RC-12Q  GUARDRAIL  COMMON  SENSOR 

The  Army’s  Guardrail  system  supports  the  tactical  commanders  by  providing 
near-real-time  SIGINT  and  targeting  information  for  deep  battle  and  follow-on  forces 
attack  support.  The  Guardrail  collects  radio  signals  from  selected  low,  mid  and  high 
bands  in  order  to  identify/classify  the  signals,  determine  source  locations,  and  provide 
near-real-time  reports  to  the  commanders  [24]. 

4.1.4.2  Space-based  Systems 

Satellite  systems  are  an  integral  part  of  ISR  operations.  Satellites  provide  nearly 
worldwide  coverage,  and  are  fairly  immune  to  enemy  actions  (at  least  for  now). 
However,  coverage  is  extremely  predictable,  and  therefore  susceptible  to  deception. 
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Weather  effects  also  hinder  some  systems.  Space  assets  provide  a  variety  of  ISR 
products  including  indications  and  warning  of  ballistic  missile  launches,  weather  and 
terrain  information,  imagery,  and  signals  intelligence.  Many  of  the  national  systems  used 
are  classified  in  name  and  capability.  Two  unclassified  sensors  are  described  below. 

4.1.4.2.1  Defense  Support  Program  (DSP) 

The  DSP  satellites  provide  the  nation’s  current  space-based  early  warning  system. 
DSP  satellites  detect  ballistic  missile  launches  around  the  world.  They  have  the 
capability  to  detect  short-range  missiles,  such  Scuds,  as  well  as  the  longer-range 
Intercontinental  Ballistic  Missiles  (ICBM).  The  DSP  system  consists  of  the  satellites,  the 
ground  processing  stations,  and  the  communication  system.  Although  the 
communication  of  Scud  launch  detections  to  the  Patriot  batteries  functioned  well  during 
Desert  Storm,  significant  improvements  in  communications  have  made  the  DSP  reports 
even  more  timely.  Early  warning  capabilities  will  continue  to  be  enhanced  with  new 
systems  being  developed  such  as  the  Space-Based  Infrared  Satellite  (SBIRS)  and  Space- 
Based  Radar. 

4.1.4.2.2  SPOT  Satellite  System 

The  SPOT  satellite  Earth  Observation  System  was  designed  by  the  ONES  (Centre 
National  d'Etudes  Spatiales),  France,  and  developed  with  the  participation  of  Sweden  and 
Belgium.  The  system  comprises  a  series  of  satellites  in  circular,  sun-synchronous  orbits 
along  with  ground  facilities  for  satellite  control  and  programming,  image  production  and 
distribution  [44]. 
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4.1.4.3  Ground-based  systems 

Some  ground-based  systems,  such  as  those  of  the  theater  air  control  system 
(TACS)  are  not  tasked  as  ISR  assets,  but  do  provide  surveillance  as  a  by-product  of  their 
primary  mission.  One  ground-based  ISR  resource  is  the  Space  Surveillance  Network 
which  tracks  and  catalogs  all  manmade  objects  in  space.  The  Ballistic  Missile  Early 
Warning  System  is  a  ground-based  system  of  radars  to  detect  strategic  missile  launches. 
HUMINT  is  gathered  by  people  such  as  Special  Operations  Forces  doing  a 
reconnaissance  mission,  aircrews,  dedicated  HUMINT  personnel,  or  prisoners.  Below 
are  three  examples  of  the  systems  used  in  ground-based  collection. 

4.1.4.3.1  Counterintelligence/Human  Intelligence  Automated  Tool  Set 


Figure  17.  CHATS 

The  Counterintelligence  /Human  Intelligence  (CI/HUMINT)  Automated  Tool  Set 
(CHATS)  can  be  operated  up  to  the  SECRET  level.  It  provides  CI/HUMINT  teams  in 
the  field  the  capability  to  “...manage  assets  and  analyze  information  collected  through 
investigations,  interrogations,  collection,  and  document  exploitation.  With  CHATS,  Cl 
units  may  electronically  store  collected  information  in  a  local  database,  associate 
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information  with  digital  photography,  and  transmit/receive  information  over  existing 
military  and  civilian  communications”  [8]. 

4.1.4.3.2  Ground  Based  Common  Sensor  (GBCS) 


Figure  18.  GBCS-H 


“Ground-Based  Common  Sensor  (GBCS  Heavy  and  Light)  GBCS  is  the  Army's 
only  on-the-ground,  all-weather,  all-terrain,  self-contained,  fully  integrated,  24-hour 
signal  intelligence  and  electronic  warfare  asset.  The  Electronic  Attack  (EA)  module 
includes  smart  jamming  capabilities.  The  GBCS  preprocesses  signal  data  at  the  sensor 
and  provides  target  detection,  identification,  and  location  reports  in  near-real-time”  [23], 
4.1.4.4  Sea-based  systems 

Dedicated  intelligence  units,  along  with  assets  that  have  other  primary  functions, 
such  as  a  destroyer  at  sea  sending  a  surface  contact  report,  conduct  naval  intelligence.  In 
fact,  naval  forces  are  unique  in  that  intelligence  collection  capabilities  are  resident  in 
many  of  their  weapons  platforms.  At  the  tactical  and  operational  levels  of  warfare, 
intelligence  collection  is  just  one  more  capability  of  ships,  submarines  and  aircraft  [19:5]. 
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For  years  the  attack  submarine  has  performed  various  ISR  roles,  and  in  the  future 
Autonomous  Underwater  Vehicles  (AUV)  or  drones  may  also  be  used  to  venture  into 
areas  unsafe  for  a  submarine.  These  AUVs  would  be  launched,  travel  into  the  area, 
perform  the  assigned  mission,  and  then  return  to  the  submarine  or  transmit  its  data  to  a 
satellite  [45]. 

4.1.5  Analyze 

After  data  has  been  collected,  it  is  processed  and  then  analyzed.  Some  sensors 
have  “on-board”  processing  capabilities,  while  others  must  send  the  raw  data  to  a 
production  center.  Processing  involves  converting  information  into  a  format  usable  by 
intelligence  personnel.  After  processing,  the  data  is  analyzed  either  at  the  production 
center  or  an  intelligence  center.  Through  analysis  and  fusion,  information  is  then 
integrated  into  the  battlefield  picture.  Fusion  is  the  result  of  comparing  and  combining 
data  from  various  collectors  along  with  previous  analysis  to  provide  an  integrated, 
hopefully  more  accurate,  understanding  of  the  true  state  of  the  object  of  collection. 
Analysis  involves  human  thinking  and  determination  in  deciding  which  information  is 
most  correct  and  useful  in  determining  the  final  conclusion.  Knowledge  of  the  enemy, 
professional  experience,  and  judgment  affect  an  analyst’s  ability  to  interpret  and  decipher 
the  significance  of  new  information,  and  determine  its  integration  with  previous 
intelligence.  When  trying  to  “fuse”  data,  analysts  consider  the  means  in  which  the  data 
was  collected,  and  its  associated  reliability  and  capability.  They  also  factor  in  the  time 
since  the  data  was  collected.  Integrating  new  data  with  other  sensor  information  and 
previous  data  is  somewhat  subjective  to  the  experience  level  of  the  analyst.  The  level  of 
work  and  the  time  allotted  to  arrive  at  a  conclusion  also  impacts  the  ability  of  analysts  to 
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accurately  fuse  the  data.  Almost  all  data  is  fused  and  analyzed  before  being  disseminated 
to  the  appropriate  agencies.  However,  as  stated  before,  some  data  can  be  processed  and 
distributed  in  near-real-time  from  ISR  assets  directly  to  the  users  bypassing  the  analysis 
and  fusion  procedure.  This  “short-cut”  intelligence  process  supports  the  “Sensor-to- 
Shooter”  concept  so  that  perishable  data  can  be  used  right  away  in  circumstances  such  as 
locating  and  killing  an  intruding  aircraft  [15:19].  This  type  of  intelligence  is  useful  at  the 
tactical  level,  although  most  operational  and  strategic  intelligence  is  most  useful  and 
reliable  after  being  analyzed  and  fused.  The  resulting  products  from  analysis  are  reports 
concerning  items  such  as  enemy  capabilities,  resources,  and  activities.  BDA  is  a  critical 
output  of  analysis  heavily  impacting  target  development  and  target  nomination 
procedures. 

Various  forms  of  intelligence  are  processed  differently  and  by  different  agencies. 
For  example,  HUMINT/CI  information  processing  primarily  involves  report  preparation 
by  personnel  within  the  J-2X.  “Further  processing  human  resource  reporting  is 
conducted  by  the  JIC  and  joint  force  analytical  and/or  production  activities;  this  primarily 
involves  analyzing  HUMINT/CI  reporting  for  inclusion  in  all-source  production  and/or 
for  data  base  maintenance”  [12:3.26]. 

The  JIC  processes  and  exploits  imagery  in  theater.  The  JIC  can  process  the 
digital  signal  and  display  it  on  a  workstation  in  softcopy  form.  The  images  can  then  be 
incorporated  into  an  all-source  product  to  aid  in  determining  the  status  of  a  target  or  other 
item  of  interest.  The  images  can  also  be  used  to  update  intelligence  databases. 

“COMINT  processing  is  accomplished  by  NSA/CSS  elements  either  assigned  to 
or  in  support  of  the  joint  force  mission.  ELINT  processing  in  support  of  a  joint  force  may 
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come  from  a  number  of  sources  including  assets  attached  to  the  joint  force,  national 
ELINT  centers,  the  JC2WC,  and  combatant  command  JICs”  [12:3.28]. 

MASINT  tends  to  be  a  processing-intensive  collection  discipline.  The  Central 
MASINT  Office  and  Service  intelligence  centers  process  MASINT  [12:3.28]. 

4.1.5.1  Processing  and  analysis  systems 

Processing  and  analysis  can  be  made  easier  through  the  use  of  computers  and 
computer  systems.  Some  of  the  systems  used  by  intelligence  personnel  are  summarized 
below. 

4.1.5.1.1  Joint  Service  Imagery  Processing  System  (JSIPS) 

JSIPS  is  a  joint  program  developed  by  the  USAF,  US  Navy  (USN),  and  US 
Marine  Corps  (USMC)  to  provide  a  common  ground  station  “...capable  of  receiving, 
processing,  exploiting,  and  disseminating  imagery  intelligence  products  collected  by 
national,  theater,  and  selected  tactical  reconnaissance  assets”  [31]. 

4.1.5.1.2  Enemy  Situation  Correlation  Element  (ENSCE) 

The  Army  and  Air  Force  are  developing  better  ways  to  coordinate  the  efficient 
use  of  surveillance/reconnaissance  systems  and  selection  of  second-echelon  targets 
through  the  use  of  automated  intelligence  fusion  centers.  “The  Army  All-Source 
Analysis  System  (AS AS)  and  the  Air  Force  Enemy  Situation  Correlation  Element 
(ENSCE)  speed  the  fusion  and  correlation  of  intelligence  sources  and  provide  a  common 
view  of  the  battlefield.  The  ASAS/ENSCE  system  will  be  an  all-source  system, 
receiving  inputs  from  tactical  Army  collection  systems  (such  as  Guardrail  and 
Quicklook),  J-STARS,  and  ATARS,  and  national  intelligence  sensors”  [34:22], 
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4.1.5.1.3  Sentinel  Byte 

JSIPS  and  ENSCE  only  provide  intelligence  to  the  headquarters  level.  To  transfer 
this  all-source  intelligence  to  the  users  at  the  wing  and  squadron  level,  the  Air  Force  is 
deploying  the  Sentinel  Byte  system.  “Sentinel  Byte  is  an  interactive  intelligence  system 
for  passing  intelligence,  targeting,  and  battle  damage  assessment  information  up  and 
down  the  chain  of  command  within  the  tactical  air  control  system”  [34:23]. 

4. 1.5. 1.4  Constant  Source 

“Constant  Source  is  a  broadcast  system  designed  to  provide  timely  intelligence  to 
combat  units  and  elements  of  the  tactical  air  control  system  at  the  secret  or  higher  level” 
[34:24].  Constant  Source  integrates  and  correlates  information  received  from  the  Navy's 
Tactical  Receive  Equipment  and  Related  Applications  (TRAP)  and  the  Tactical  Data 
Information  Exchange  System  -Broadcast  (TADIX-B)  including  data  from  national 
systems  [7]. 

Constant  Source  correlates  reports  and  compares  new  information  with  already 
existing,  or  known,  information.  It  updates  “...data  bases,  refines  emitter  locations,  and 
identifies  moving  targets  and  movements  of  known  sites”  [34:24]. 

Constant  Source  can  also  filter  data  according  to  the  user’s  desires.  The  display 
can  provide  color  graphics  of  correlated  tracks  overlaid  on  a  map.  An  alarm  is  used  to 
alert  the  user  to  high-interest  events,  and  the  user  may  watch  events  in  near-real-time  as 
well  as  review  past  events. 

4.1.6  Dissemination 

The  joint  intelligence  architecture  allows  collectors,  producers,  and  users  of 
intelligence  to  be  interconnected  and  share  information.  Interoperable  systems  link 
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intelligence  agencies  at  all  levels  -  theater  JICs,  JISEs,  Service  intelligence,  and  national 
organizations. 

Information  is  only  useful  if  those  who  require  it  actually  receive  it. 

Dissemination  is  the  distribution  of  intelligence  to  users  in  a  suitable  form.  It  is  a  critical 
step  as  timely  dissemination  of  information  is  necessary  for  optimal  decision-making. 
Decisions  can  only  be  based  upon  previous  knowledge  and  relevant  new  information  that 
has  been  received.  If  new  information  does  not  arrive  in  time  to  provide  a  better 
understanding  of  the  current  situation,  decisions  must  be  made  somewhat  in  ignorance. 
This  can  cost  money,  resources,  and  human  lives. 

Dissemination  comes  is  a  variety  of  forms  and  includes  physical  transfer  of 
information,  digital  and  analog  media,  video-teleconference,  telephone,  FAX  machine, 
remote  access  to  data  bases,  radio,  satellite  broadcasts,  etc.  However,  the  aim  of 
dissemination  is  not  to  overwhelm  the  user  with  massive  amounts  of  data.  The  concept 
for  dissemination  today  is  a  push/pull  methodology  that  emphasizes  pushing  intelligence 
to  the  warfighter  (through  over-the-air  updates)  and  allows  the  warrior  to  pull  information 
on  demand  [1 1 :2.7].  Dissemination  should  provide  an  optimum,  rather  than  a  maximum, 
amount  of  information. 

The  Defense  Intelligence  Dissemination  System  (DIDS)  manages  the  push 
concept.  Before  “pushing”  information,  the  producer  queries  the  DIDS.  The  pull 
concept  involves  accessing  information  through  databases,  files,  or  other  means  used  by 
intelligence  organizations.  “Pull”  products  are  available  through  a  number  of  ways, 
including  INTELINK  and  the  GBS. 


56 


While  hardcopy  distribution  of  intelligence  is  possible,  with  new  technology  and 
the  need  to  improve  timeliness,  most  intelligence  products  are  disseminated  in  electronic 
form.  Some  of  the  communication  tools  used  to  share  information  are  Joint  Worldwide 
Intelligence  Communications  System  (JWICS),  Joint  Deployable  Intelligence  Support 
System  (JDISS),  DoD  Intelligence  Information  System  (DoDIIS),  Open  Source 
Information  System  (OSIS),  Global  Command  and  Control  System  (GCCS),  INTELINK, 
and  GBS  [12:3.43].  “The  joint  intelligence  architecture  uses  JWICS  and  JDISS  as  the 
joint  standard  and  foundation  for  commonality  among  support  systems”  [11:7.5].  The 
GCCS  and  GBS  were  presented  earlier  in  this  report;  the  rest  of  the  tools  are  summarized 
below. 

4.1.6.1  Department  of  Defense  Intelligence  Information  System  (DODIIS) 

DODIIS  comprises  the  worldwide  inter-computer  network  linking  Intelligence 
Data  Handling  Systems  (IDHS).  It  encompasses  the  intelligence  storage  and  retrieval 
devices  (IDHS),  the  transmission  means  (JWICS)  and  the  interface  (JDISS  or  other 
computer  interface  systems). 

DODIIS  provides  input  processing  for  a  variety  of  intelligence  products, 
including  “...imagery  exploitation,  ELINT,  COMINT  and  HUMINT,  as  well  as, 
intelligence  data  development,  target  material  production,  target  data  development  and 
scientific  and  technical  intelligence”  [14:7.34-35]. 

4.1.6.2  Joint  Deployable  Intelligence  Support  System  (JDISS) 

The  JDISS  allows  for  connectivity  and  interoperability  with  the  intelligence 
systems  at  the  headquarters  level  as  well  as  the  deployed  units.  JDISS  has  evolved  into  a 
widely  accepted  intelligence  workstation  standard  and  is  the  “...technical  baseline  for  the 
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DoDIIS  client/server  environment”  [29].  It  provides  the  JIC,  JTF  and  operational 
commanders  with  on-site  automation  support  such  as  transmitting  and  receiving  specific 
requests  for  intelligence,  accessing  databases,  supporting  digitized  imagery  exchange, 
and  performing  multi-media  functions  such  as  electronic  publishing  and  video 
teleconferencing.  “Based  on  a  SunSparc  workstation  with  an  open  systems  architecture, 
JDISS  is  equipped  with  a  core  set  of  software  applications  that  give  the  intelligence 
analyst  access  to  a  large  number  of  intelligence  databases. .  .as  well  as  the  ability  to 
perform  independent  multi-disciplined  intelligence  analysis  in  the  field”  [14:7.32]. 

4.1.6.3  Joint  Worldwide  Intelligence  Communications  System 

JWICS  is  the  SCI  portion  of  DISN.  Owned  and  operated  by  DIA,  it  provides 
DODIIS  users  a  SCI-level  high-speed  multimedia  network  using  high-capacity 
communications  to  handle  data,  voice,  imagery,  and  graphics.  The  system  uses  JDISS  as 
its  primary  means  of  operator  interface  and  display.  “In  addition  to  being  a 
communications  system,  JWICS  provides  secure,  interactive  video  teleconferencing  to 
the  members  of  the  DoD  Indications  and  Warning  system  at  Unified  Commands  and 
service  headquarters  within  the  US  and  overseas.  This  system  enables  Indications  and 
Warning  centers  to  share  information  with  other  watch  centers  throughout  DoD” 
[14:7.36-37].  It  is  used  to  broadcast  daily  and  /or  crisis  intelligence  briefings  from  any 
one  site  to  one  or  more  sites  [10:69]. 

4.1.6.4  Open  Source  Information  System  (OSIS) 

The  Open  Source  Information  System  (OSIS)  consists  of  an  unclassified  group  of 
systems  serving  the  intelligence  community  with  open  source  intelligence.  Community 
Open  Source  Program  Office  (COSPO)  supports  all  aspects  of  open  source  information 
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systems,  spanning  collection,  processing,  analysis,  and  dissemination,  to  include  network 
and  distributed  computing  resources.  The  contents  of  OSIS  includes  the  Central 
Information  Reference  and  Control  Database  of  over  10  million  titles  on  scientific  and 
technical  topics,  including  patents,  standards,  military  equipment  and  systems; 
Conference  Database  of  upcoming  symposia,  congresses,  and  conventions  in  the  areas  of 
science,  technology,  engineering,  politics,  and  economics;  Digital  Terrain  Elevation  Data 
map  collection  (from  NIMA)  providing  global  coverage;  Foreign  Broadcast  Information 
Service  products  including  the  Daily  Reports,  Science  &  Technology  Perspectives, 
Trends,  and  Pacific  Rim  Economic  Review;  abstracts  and  complete  articles  on 
telecommunications  related  topics;  and  Technical  Equipment  List  indexes  with  over 
100,000  brochures  and  manuals  on  telecommunications  and  related  equipment  [38]. 
4.1.6.5  INTELINK 

“Intelink,  which  began  testbed  operation  in  1994,  is  both  an  architectural 
framework  and  an  integrated  intelligence  dissemination  and  collaboration  service 
providing  uniform  methods  for  exchanging  intelligence  among  intelligence  providers  and 
users”  [26].  Patterned  after  the  Internet,  Intelink  provides  a  global  network  to  allow  the 
sharing  of  documents  and  other  resources  among  numerous  intelligence  agencies  (e.g. 
FBI,  CIA,  DEA,  NS  A,  NRO). 

4.1.7  Evaluate 

After  receiving  the  intelligence,  the  user  evaluates  it  to  ensure  it  satisfies  the 
requirement.  A  requirement  is  most  often  considered  satisfied  if  the  “...intelligence 
provided  to  the  requestor  is  complete,  timely,  and  in  a  usable  format”  [12:3.2].  If  the 
intelligence  is  satisfactory,  the  user  may  provide  feedback  to  ensure  that  the  ISR  process 
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continues  to  fulfill  the  user’s  requirements.  If,  after  evaluation,  the  user  is  not  satisfied, 
the  user  must  convey  the  dissatisfaction,  and  part,  or  all,  of  the  ISR  process  may  have  to 
be  re-accomplished.  Dissatisfaction  can  result  from  the  information  being  not  what  was 
asked  for,  not  to  the  degree  of  resolution  needed,  or  not  timely  enough  to  be  of  use. 

Air  Force  Instruction  14-201  [16:12]  gives  an  example  Requirement  Satisfaction 
report  providing  areas  for  evaluation  and  a  rating  scale  as  seen  in  Table  3. 


Table  3.  Requirement  Satisfaction 


Areas  for  Evaluation 

Rating  Values  (scale  from  0-6) 

1 .  Timeliness 

2.  Objectivity 

3.  Usability 

4.  Readiness 

5.  Completeness 

6.  Accuracy 

7.  Relevance 

0.  Totally  unacceptable 

1 .  Poor 

2.  Marginal 

3.  Fair 

4.  Good 

5.  Excellent 

6.  Outstanding 

4.1.8  Apply 

Upon  satisfaction  of  the  intelligence  received,  the  final  step  in  the  ISR  process  is 
the  application  of  the  product.  Application  can  come  in  many  different  forms.  For  near- 
real-time  collection,  intelligence  can  be  applied  by  a  pilot  to  re-strike  a  target.  Situation 
displays  providing  information  to  a  variety  of  users  can  be  updated  to  inform  them  of 
enemy  locations.  Briefings  to  decision-makers  may  also  be  the  result  of  an  ISR  product. 
Each  requirement  has  its  own  application,  and  the  ISR  product  must  meet  the  needs  of  the 
user  to  ensure  application  is  viable. 
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4.2  Summary 

The  ISR  process  is  a  continual,  somewhat  complicated  cycle,  vital  to  military 
success.  Many  organizations,  assets,  and  personnel  are  involved  in  making  the  process 
run  smoothly  and  efficiently.  The  coordination  and  communication  needed  among  ISR 
personnel,  warfighters,  and  decision-makers  is  essential  to  ensure  the  optimal  level  of 
information  is  provided  and  is  useable.  ISR  operations  will  only  continue  to  increase  in 
importance  as  information,  and  information  superiority,  become  more  critical  elements  in 
conducting  war. 
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5.  THUNDER  Intelligence.  Surveillance,  and  Reconnaissance  Methodology 


5.1  Introduction 

Intelligence,  Surveillance,  and  Reconnaissance  is  a  fairly  recent  addition  to  the 
THUNDER  model.  Previous  chapters  have  covered  the  actual  ISR  process.  This  chapter 
will  demonstrate  how  THUNDER  has  implemented  ISR  in  order  to  capture  this  process. 
THUNDER  represents  ISR  effects  using  parameters  such  as  processing  time,  probability 
of  coverage,  and  perception,  along  with  modeling  various  ISR  resources  such  as 
satellites,  airborne  assets,  and  ground  based  assets.  The  THUNDER  ISR  module  is 
designed  to  capture  the  effects  of  “...information  collection  on  air  planning,  sensor  and 
target  performance  attributes,  requirement  for  and  value  of  Battle  Damage  Assessment 
(BDA),  and  processing  delay  and  timeliness  of  information”  [54]. 

5.2  Levels  of  ISR 

THUNDER  has  three  levels  of  ISR  resolution:  Low,  High  and  Very  High.  The 
user  specifies  each  side’s  ISR  level.  The  names  used  for  the  levels  of  resolution  can  be 
slightly  misleading,  as  they  do  not  in  any  way  indicate  the  quality  of  ISR  provided.  Low 
resolution  is  equivalent  to  “perfect”  intelligence.  Essentially,  Low  resolution  does  not 
show  the  uncertainty  of  ISR  effects  since  the  side  has  perfect  knowledge  of  the  enemy. 

In  High  and  Very  High  resolutions,  decisions  are  based  on  perceptions  of  the  truth.  The 
quality  of  these  perceptions  may  degrade  over  time  until  new  observations  are  made. 

The  main  distinction  between  High  and  Very  High  is  in  the  detail  of  the 
perceptions  generated.  High  resolution  maintains  perceptions  on  the  enemy’s  zone/sector 
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areas,  while  Very  High  maintains  perceptions  on  specific  enemy  targets.  High  resolution 
intelligence  comes  from  aircraft  assets  while  Very  High  intelligence  also  involves 
satellites  and  scripted  observations.  Scripted  observations  may  represent  any  intelligence 
source  desired  by  the  user,  such  as  army  ground  sensors  or  HUMINT.  The  most  realistic 
scenario  entails  using  Very  High  resolution,  and  thus,  this  study  only  examines  and 
explains  Very  High  I  SR  methodology. 

Two  THUNDER  ISR  terms  need  differentiating:  perception  and  confidence.  A 
simple  example  illustrates  what  these  terms  mean.  When  a  sensor  makes  an  observation, 
a  random  draw  is  made  for  the  perception  of  a  target.  A  confidence  level  is  given  by  a 
user-input  default  value.  The  confidence  level  of  a  sensor  inversely  affects  the  variability 
of  the  perception  -  the  higher  the  confidence,  the  less  variability  from  the  truth.  If  a 
target  has  10  tanks,  the  random  draw  may  cause  the  sensor  to  only  perceive  7  tanks. 
However,  the  confidence  may  be  90%  that  there  are  7  tanks.  As  time  goes  on,  the 
confidence  that  there  are  7  tanks  may  degrade  to,  say,  80%.  When  planning  for  air-to- 
ground  strikes,  the  ATO  generator  selects  sortie  allocation  and  aircraft/munition 
configuration  by  using  the  highest  confidence  level  for  each  target  attribute  among  all 
sensor  observations  of  the  target.  A  small  example  of  how  this  works  is  seen  in  Table  4. 
This  methodology  is  an  attempt  at  fusing  data  from  more  than  one  source  of  information. 
THUNDER  uses  the  data  with  the  highest  confidence,  even  though  this  perceived  data 
may,  in  fact,  be  incorrect. 


63 


Table  4.  Example  of  “Fusing”  Sensor  Reports 


Sensor 

Target  Attribute 

Report 

Confidence 

“A” 

Number  of  tanks 

10 

90% 

“B” 

Number  of  tanks 

20 

80% 

“A” 

Velocity 

5 

75% 

“B” 

Velocity 

15 

85% 

Used  for  Planning 

Number  of  tanks 

10 

Used  for  Planning 

Velocity 

15 

5.3  ISR  Sensors/Platforms 

ISR  sensors  and  their  capabilities  are  defined  and  differentiated  through  input  data 
parameters.  The  parameters  that  identify  each  type  of  sensor  include: 

-  Real-time  or  batch  sensor  delivery.  As  expected.  Real-time  indicates  that  the 
sensor  can  pass  intelligence  data  over  a  communications  link  in  a  real-time 
manner.  Batch  delivery  represents  sensors  that  do  not  disseminate  data  in  a 
real-time  manner  and  therefore  must  physically  “deliver”  the  data  before 
processing  can  begin. 

-  Processing  time.  Time  to  process  and  analyze  raw  data  is  represented  by  a 
probability  distribution. 

-  Night  Capable.  Indicates  if  the  sensor  is  able  to  gather  information  at  night. 

-  Min/Max  Sensor  Range.  Denotes  the  observation  range  limitation  of  the 
sensor.  Not  used  for  standoff  reconnaissance  missions. 

-  Two  Sigma  Target  Location  Error.  The  sensor  will  report  the  location  of  a 
target  with  this  random  error. 

-  Maximum  Number  of  Targets  Per  Sorties/Pass.  The  sensor  will  observe  a 
maximum  number  of  the  highest  priority  targets  during  a  pass. 

-  Sensor  Type  (Grid  or  FLOT).  This  parameter  only  has  significance  for 
scripted  events.  A  sensor  denoted  as  “Grid”  will  only  look  in  the  specified 
Grid  locations;  the  ISR  Grid  is  defined  by  the  user.  A  “FLOT”  sensor  will 
scan  along  the  FLOT  (Forward  Line  of  Own  Troops)  given  its  min/max  range. 

-  Overfly  the  Target.  Denotes  whether  the  sensor  must  overfly  the  target  in 
order  to  make  an  observation. 
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-  Observe  Through  Clouds.  Indicates  whether  cloud  coverage  will  inhibit  the 
sensor’s  ability  to  gather  information  on  a  target. 

-  Standoff  Range.  Distance  from  the  target  in  which  a  standoff  reconnaissance 
sensor  can  make  an  observation.  Only  used  for  standoff  sensors. 

-  Perception  Update  Interval.  Time  between  observations  for  a  sensor  hovering 
over  an  area  or  site  of  interest. 

Any  aircraft,  satellite,  or  scripted  event  identified  as  having  an  ISR  sensor  may 
collect  intelligence  data.  Aircraft  assets  typically  modeled  in  THUNDER  include  J- 
STARS,  AW  ACS,  U-2,  and  UAV.  Any  flight  group  carrying  appropriate  sensors  may 
conduct  BDA  upon  execution  of  an  air-to-ground  attack.  Satellites  are  described  by 
orbital  parameters  that  determine  when  the  satellite  will  be  in  coverage  and  the  range  of 
that  coverage.  Scripted  events  are  usually  identified  as  either  Grid  or  FLOT  sensor 
events  and  given  a  probability  distribution  to  generate  times  between  observations.  Grid 
sensor  events  are  defined  as  observations  made  in  certain  grid  cells  of  the  ISR  grid. 
FLOT  sensor  events  are  observations  made  within  the  maximum  range  of  the  sensor, 
along  the  FLOT.  Aircraft  can  carry  only  one  ISR  sensor  whereas  satellites  or  scripted 
events  may  be  defined  as  having  more  than  one  sensor.  Figure  19  shows  the  various 
types  of  intelligence  platforms  that  can  be  used  in  THUNDER  and  a  sample  coverage 
area  each  provides. 
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Figure  19.  THUNDER  ISR  Platforms 


5.4  Target  Classes 

THUNDER  groups  the  targets  in  a  scenario  in  three  different  ways  for  ISR:  ISR 
target  classes,  planning  classes,  and  perception  classes. 

Air-to-ground  targets  are  given  an  ISR  target  class  which  categorizes  that  target 
according  to  characteristics  such  as  whether  it  is  fixed,  mobile,  engaged,  moving,  etc. 
Each  class  has  a  “Time  vs.  Probability  of  Movement  Curve”  and  a  “Time  vs.  2-Sigma 
Location  Latency  Error  Curve.”  These  curves  impact  whether  or  not  a  flight  group 


acquires  a  ground  target.  Recall,  when  a  sensor  observes  a  target,  it  already  has  a  2- 
sigma  location  error.  When  a  flight  group  reaches  the  target  area,  whether  or  not  the 
flight  group  acquires  the  target  depends  on  the  time  since  the  last  observation  and  curves 
defined  for  the  ISR  class  containing  the  target.  For  example,  when  a  sensor  gathers  data 
about  a  mobile  target,  it  reports  a  location  that  has  a  random  error,  defined  by  the  sensor, 
attached  to  it.  When  the  flight  group  reaches  the  target  area,  the  location  it  goes  to  has  an 
additional  error,  defined  by  the  target  class,  since  the  target  has  probably  moved.  See 
Figure  20  for  graphical  depiction. 


Observation  of  Target 


Perceiyed  Target  Location 


Target  Acquisition 


Aircraft  tries  to  acquire 
at  this  location 


Target  moves 


Error 

■ 

Perceived  location 
from  satellite 

True  Target  Location  I  observation  True  Target  Location 

Figure  20.  Impact  of  ISR  Sensor  Target  Location  Error  on  Target  Acquisition 
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Standard  targets  are  also  assigned  a  planning  class.  Planning  classes  allow  the 
user  to  impact  how  targets  are  prioritized  for  reconnaissance  missions  as  well  as 
influence  nomination  rules  for  air-to-ground  attack.  Figure  21  shows  the  process  of 
prioritizing  targets  for  collection.  Targets  that  are  assigned  to  a  particular  planning  class 
have  the  following  similar  parameters  for  target  prioritization. 

-  Options  if  BD A  has  not  been  accomplished  for  the  target:  1 )  assume  the  target 
is  still  live  and  therefore  use  the  previous  perception,  2)  assume  the  target  has 
been  damaged  and  use  a  degrade  (multiplier  also  defined  for  each  class),  or  3) 
assume  the  target  is  dead  and  do  not  nominate  for  air-to-ground  attack. 

-  Distinct  priority  multipliers  for  reconnaissance  target  nomination  given  that 
the  target  has  been  nominated  for  air-to-ground  attack,  has  not  been 
nominated,  and  has  not  been  observed  since  it  was  last  attacked. 

-  Minimum  priority  that  the  target  can  be  assigned  when  planning  for 
reconnaissance/surveillance  missions. 

-  Confidence  level  curve  that  defines  the  reconnaissance  priority  multiplier  as  a 
function  of  the  confidence  level  for  the  target.  For  example,  one  may  want  to 
ensure  that  we  always  have  80%  for  a  target,  and  therefore,  if  the  confidence 
is  lower  than  80%,  the  priority  multiplier  will  increase  the  target’s  priority  for 
a  reconnaissance  mission. 

Targets  are  also  assigned  to  an  ISR  perception  class.  This  defines  the 
probabilities  that  a  live  target  is  perceived  as  live.  The  perception  class  also  defines 
default  values  for  the  confidence  levels  when  an  observation  is  made  of  the  target,  as  well 
as  the  confidence  degradation  curve  as  a  function  of  time  since  the  observation.  The 
Probability  of  Coverage  is  also  defined  in  this  class.  The  probability  of  coverage  can  be 
used  to  represent  deception  techniques  such  as  targets  being  camouflaged. 
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Figure  21.  THUNDER  ISR  Target  Prioritization 


5.5  Intelligence  Collection 

Only  aircraft  reconnaissance  (RECCE)  and  standoff  reconnaissance  (SREC) 
missions  are  automated  in  the  intelligence  collection  process.  The  user  must  preplan  any 
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other  type  of  collection,  such  as  satellite  observations  or  scripted  observations.  A 
satellite  is  assumed  to  always  collect  observations  when  the  theater  is  in  its  view. 
Likewise,  anything  scripted  by  the  user  is  collected  during  the  scripted  times. 

A  target’s  priority  for  a  reconnaissance  mission  is  based  upon  its  priority  for 
strike  missions.  The  strike  priority  is  adjusted  using  the  multipliers  in  the  planning  class 
(see  “Target  Classes”  above)  to  determine  its  reconnaissance  priority.  Reconnaissance 
missions  are  planned  just  as  strike  missions  are  planned. 

Reconnaissance  can  also  be  obtained  during  an  air-to-ground  attack  with  aircraft 
carrying  the  appropriate  sensors,  representing  either  electronic  or  human  reports.  Figure 
22  shows  the  collection  process. 

5.6  ISR  Nodes 

ISR  nodes  are  entities  in  the  ISR  process  that  can  affect  the  processing  time  of  an 
observation,  e.g.  a  communications  center.  The  ISR  sensors  are  grouped  into  sensor 
classes,  and  the  sensors  in  a  particular  class  share  the  same  processing  delay  if  ISR  nodes 
are  no  longer  functioning.  They  also  specify  the  probability  of  an  observation  being  lost 
if  ISR  nodes  are  damaged/destroyed.  An  example  of  a  sensor’s  processing  rules  is  shown 
in  Figure  23.  The  Rule  Sequence  checks  the  status  of  the  ISR  Node, 

Blue_Comm_Center.  If  all  of  the  Blue_Comm_Centers  have  been  destroyed,  the 
processing  delay  of  the  observation  increases,  and  a  probability  that  the  observation  is 
lost  is  also  implemented.  The  processing  delay  can  be  adjusted,  however,  if  a  Blue 
AW  ACS  is  still  live.  The  whole  ISR  observation  process  is  depicted  in  Figure  24. 
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Figure  22.  Intelligence  Collection 


@Grid  and  Flot  Sensors 
10001 
BLUE 

BEGIN . ISR. PROCESSING . RULE . SEQUENCE 
IF  COUNT (Blue_Comm_Center)  =  0 
THEN 

LET  PROCESSING. DELAY  =  2.0  *  PROCESSING. DELAY 
LET  PROB. LOST. OBSERVATION  =0.25 
ENDIF 

IF  COUNT (Blue_AWACS)  >  0 
THEN 

LET  PROCESSING. DELAY  =  PROCESSING. DELAY  -  0.25 
ENDIF 
EXIT 

END. ISR. PROCESSING . RULE . SEQUENCE 


Figure  23.  ISR  Processing  Rules  Example 
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5.7  Target  Perceptions 

ISR  is  used  in  THUNDER  to  generate  perceptions  of  true  target  attributes.  These 

perceptions  affect  mission  planning  and  mission  effectiveness. 

At  the  time  of  each  sensor  observation,  random  draws  are  made  to 
generate  a  specific  target  perception  with  specific  levels  of  confidence  for 
each  of  the  target  attributes  (size,  location,  velocity,  etc.).  The  confidence 
levels  for  that  observation  are  then  degraded  with  the  passage  of  time.  At 
any  point  in  time,  the  ISR  view  of  a  target  will  consist  of  a  vector  of  target 
attribute  perceptions  built  by  ‘fusing’  the  observations  from  reporting 
sensors.  Fusion  is  modeled  by  taking,  for  each  attribute,  the  highest 
degraded  confidence  level  across  sensors  and  using  that  confidence  level 
to  represent  the  level  of  perception  [51:135]. 

Each  time  a  sensor  makes  another  observation  on  a  target,  its  old  perception  is  discarded. 
The  perceived  status  of  each  target  includes  the  following  parameters: 

-  Tons  of  supplies; 

-  Force  ratio; 

-  Unit  strength; 

-  Message  processing  capability; 

-  Number  of  spares; 

-  Numbers  of  aircraft  (by  type)  at  an  airbase; 

-  Usable  runway  length; 

-  Locations; 

-  Velocities; 

-  Arc  throughput; 

-  Air  defense  site  status; 

-  Transshipment  capacity; 

-  Logistics  facility  issue  capacity; 
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Figure  24.  ISR  Observation  Processing 


The  perception  is  only  updated  after  the  processing  time  has  passed.  The 
processing  time  can  be  affected  by  the  status  of  ISR  nodes.  Generally,  as  ISR  nodes  are 
degraded,  processing  time  increases.  This  helps  represent  decreased  bandwidth  when  an 
antenna  is  destroyed,  as  well  as  the  extra  time  needed  because  of  the  reduced  number  of 
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analysts  deciphering  the  data.  The  ISR  Processing  Rule  System  also  allows  for  a 
probability  that  an  observation  is  lost,  and  therefore  discarded.  The  probability  of  an 
observation  being  lost  could  model  factors  such  as  data  not  being  received,  data 
saturation  of  the  receiver,  or  analyst  oversight. 

5.8  ISR  Effects  on  Mission  Planning  and  Mission  Effectiveness 

Mission  planning  entails  target  nomination,  prioritization,  sortie  allocation,  and 
aircraft-munition  configuration.  THUNDER’S  air  planning  function  uses  target 
perceptions  when  formulating  the  target  list.  Nomination  rules  are  set  for  each  ISR  target 
class.  Targets  must  be  nominated  to  be  attacked.  An  example  of  this  type  of  rule  is,  if  an 
observation  has  been  made  of  the  target  within  the  last  48  hours,  then  the  target  is 
nominated. 

Different  types  of  targets  are  prioritized  using  various  formulas,  but  the 
parameters  in  these  formulas,  such  as  unit  strength,  etc. ,  are  all  based  on  their  perceived 
values;  therefore,  the  priorities  of  targets  are  based  on  the  perceptions  generated  by  the 
ISR  sensors. 

Once  a  prioritized  target  list  is  generated,  aircraft  sorties  and  their  missions  are 
assigned  according  to  the  perceived  state  of  the  targets.  The  aircraft/munition 
configuration  for  each  sortie  is  selected  based  upon  the  forecasted  weather,  as  well  as  the 
perceived  state  of  the  target. 

Target  acquisition  depends  on  the  sensor’s  target  location  error  and  how  old  the 
last  observation  is,  since,  for  example,  mobile  targets  may  have  moved.  The  probability 
of  acquiring  the  target  depends  on  the  true  distance  between  the  aircraft  and  target.  If  the 
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target  exceeds  a  maximum  distance,  it  is  not  acquired.  However,  an  aircraft  will  have 
another  chance  to  acquire  the  target  is  a  standoff  reconnaissance  aircraft  has  the  target  in 
its  view.  The  probability  that  the  aircraft  will  acquire  a  target  in  this  situation  is  given  by 
a  user-defined  parameter. 

Given  target  acquisition,  the  effectiveness  of  a  strike  depends  on  the  munitions 
allocated  to  the  mission.  This  mission-to-target  allocation  depends  on  target  perception, 
which  of  course  depends  on  the  ISR  observations. 

5.9  Effects  on  Ground  War 

The  ground  war  is  deterministic  and  highly  aggregated  in  THUNDER;  individual 
battles  do  not  actually  occur. 

Ground  units,  usually  of  division  or  brigade  size  for  on-line 
combat  units,  engage  in  combat  along  the  Forward  Line  of  [Own] 

Troops  (FLOT)  and  combat  is  adjudicated  by  the  USA  CAA’s 
Attrition  Calibration  (ATCAL)  model... FLOT  movement  is  based 
directly  on  the  relative  losses  of  both  sides,  the  postures  of  the  attacker 
and  defender,  and  the  terrain  upon  which  the  combat  occurred  [10:23]. 

Ground  units  in  battle  engagements  move  strictly  back  and  forth  within  battlefield 

sectors.  The  ISR  process  influences  ground  battles  and  the  movement  of  the  FLOT 

through  a  multiplier  on  the  lethality  of  indirect  fire  weapons  and  the  target  availability  of 

direct  fire  weapons  in  equations  used  by  ATCAL.  This  multiplier  is  activated  whenever 

a  standoff  reconnaissance  aircraft  has  coverage  of  an  enemy  unit.  The  user  inputs  a 

maximum  multiplier  indexed  by  weapon,  target,  and  posture.  “The  actual  multiplier  is 

scaled  according  to  a  user  input  curve  that  relates  SREC  aircraft  coverage  (as  a  fraction 

of  the  ground  combat  cycle)  to  the  maximum  possible  effect”  [51 :23]. 
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To  understand  how  this  multiplier  influences  ground  combat  results  and  to  get  a 
better  idea  of  its  significance,  we  need  to  look  at  the  ATCAL  process  used  in  THUNDER 
to  determine  battle  outcomes.  Figure  26  above  shows  the  general  process;  individual 
elements  of  the  process  are  further  explained  independently. 

The  SREC  multiplier  is  computed  in  the  following  manner.  First,  the  fraction  of 
the  ground  combat  cycle  that  the  target  was  in  SREC  coverage  is  determined.  Processing 
time  and  probability  of  an  observation  being  lost  are  ignored  in  this  calculation.  Second, 
the  user-input  maximum  multiplier  for  the  appropriate  weapon/target/posture 
combination  is  retrieved.  The  actual  multiplier  is  then  given  by  the  following  equation. 

SREC  Multiplier  =  1  +  [SREC  Coverage  *  (Maximum  Multiplier  - 1)]  (1) 

Prioritizing  the  targets  involves  multiplying  the  probability  of  kill  by  its 
importance.  No  SREC  effect  is  seen  in  this  calculation. 

For  direct  fire  weapons,  the  effect  of  the  SREC  multiplier  is  seen  in  the 
availability  of  targets.  It  is  a  direct  multiplier  on  availability,  which  in  turn  is  used  in 
computing  losses.  The  procedure  for  computing  losses  for  direct  fire  weapons  is  seen  in 
Figure  27  where  the  “Availability”  factor  represents  the  availability  with  the  SREC 
multiplier  applied.  Note  that  “Non-availability  of  higher  priority  targets”  does  not  use 
this  “Availability”  factor,  nor  does  the  calculation  for  Shots  Fired.  We  can  see  that  as  the 
Availability  increases,  the  losses  to  each  target  will  also  increase. 


77 


Figure  27.  Computing  Losses  to  Targets  of  Direct  Fire  Weapons 


Figure  26  shows  that  the  target  priorities  for  indirect  fire  weapons  are  weighted. 
The  number  of  rounds  fired  involves  the  weighted  priorities  and  a  bias  factor,  but  that 
number  is  not  affected  by  the  SREC  multiplier.  The  SREC  multiplier  only  affects  the 
lethality  of  the  shooter,  which  is  found  in  the  computation  of  losses.  Losses  are 
calculated  as: 

Losses  =  Rounds  fired  x  Lethality  x  Adjusted  Priority  (2) 

Note  the  Adjusted  Priority  is  used  to  divide  the  effects  of  the  rounds  among  the  targets 
based  on  priority  -  it  serves  the  same  purpose  of  target  availability  for  direct  fire 
weapons. 
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All  of  the  losses  are  then  totaled,  and  averaged  by  equipment  type.  If  the 
fractional  change  of  each  type  of  equipment  is  not  less  than  a  user-stated  e,  re-compute 
(essentially,  fight  the  war  again  for  this  cycle).  Otherwise,  assess  losses,  compute  the 
consumption,  and  determine  which  way  each  FLOT  segment  should  move. 

FLOT  movement  depends  on  unit  attrition,  unit  posture,  terrain,  POL,  unit  tactical 
march  rates,  and  opposing  command  objectives.  The  SREC  multiplier  affects  FLOT 
movement  through  the  attrition  of  units. 

The  influence  of  ISR  on  the  ground  war  presented  above  affects  units  actually  in 
combat.  ISR  also  influences  reserve  units,  supplies,  or  any  other  support  to  combat  units 
by  influencing  the  air  war  and  air  interdiction  efforts  on  lines  of  communication  and 
resources,  as  well  as  targeting  equipment  and  units  involved  in  the  ground  war. 

5.10  Summary 

This  chapter  has  presented  the  parameters  and  processes  used  by  THUNDER  to 
capture  ISR  elements  and  ISR  effects.  The  parameters  represent  the  uncertainty  of 
intelligence  data.  The  processes  show  the  compounding  effects  of  ISR  by  demonstrating 
its  influence  on  air  targeting  and  ground  unit  attrition,  which  impact  mission 
effectiveness. 
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6.  Analysis  of  the  THUNDER  ISR  Module 


6.1  Introduction 

THUNDER  has  an  ISR  module  because  of  the  crucial  impact  of  ISR  to 
warfighting  capabilities.  Previous  chapters  have  described  the  basic  components  of  the 
ISR  process  and  the  design  of  the  THUNDER  ISR  module.  This  chapter  compares  these 
two  subjects,  real  versus  modeled,  in  order  to  illustrate  which  ISR  components  have  been 
captured,  aggregated,  or  neglected  by  THUNDER. 

Bear  in  mind  that  THUNDER  is  a  campaign  level  model  and  aggregation  of 
elements  is  something  that  is  accepted  and  necessary  in  order  to  maintain  reasonable  run¬ 
times.  So,  although  some  elements  of  ISR  may  be  identified  as  missing,  or  aggregated, 
that  is  not  necessarily  something  undesirable.  THUNDER  justifiably  does  not  attempt  to 
model  the  detail  of  engagement  or  mission-level  models.  However,  as  technology 
advances,  it  may  be  possible,  and  maybe  somewhat  desirable,  to  implement  more  detail 
into  campaign-level  models,  even  if  just  for  a  few  processes.  Higher  resolution  processes 
allow  for  more  flexibility,  and  for  fewer,  or  maybe  more  valid,  assumptions.  Identifying 
the  details  of  a  process  is  a  necessary  step  before  an  attempt  at  higher  resolution  can  be 
accomplished.  Also,  knowing  which  elements  of  the  ISR  process  have  been  aggregated, 
or  dismissed,  may  allow  for  higher  resolution  models  to  either  be  identified,  or  built,  in 
an  effort  to  generate  more  accurate  input  data. 

This  chapter  will  present  the  comparison  of  the  THUNDER  ISR  process  and  the 
“real”  ISR  process  in  an  outline  similar  to  chapter  4. 
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6.2  Implementation  of  ISR  Purposes 

The  ISR  portion  in  THUNDER  has  four  basic  applications,  or  purposes,  each  of 
which  fit  into  the  fundamental  purposes  of  intelligence  identified  in  Joint  Pub  2-0.  These 
applications  are: 

-  Aid  in  aircraft/munition  configuration  decisions  for  striking  targets 

-  Report  target  status  after  air  strikes  (BDA) 

-  Assist  in  real-time  target  location  if  strikers  cannot  locate  the  target 

-  Act  as  a  force  multiplier  for  ground  units 

Figure  28  maps  THUNDER’S  ISR  applications  to  the  intelligence  purposes 
identified  in  Joint  Pub  2-0  (see  p.  1 9  for  purpose  descriptions).  Further  explanation  of  the 
comparison  follows. 

Supporting  the  commander  is  not  necessarily  a  quantitative  concept  that  can  be 
reproduced  in  a  model.  ISR  in  THUNDER  supports  aircraft  targeting  decisions  through 
target  observations  and  BDA.  Reconnaissance  or  stand-off  reconnaissance  missions  are 
not  scheduled  by  considering  any  effects  on  ground  troops,  although  these  missions  can 
be  scripted  by  the  user.  Knowledge  of  the  situation  for  any  other  type  of  decision,  such 
as  air  apportionment,  are  to  some  extent  pre-set  by  the  user  through  numerous  input  files 
when  designing  the  scenario. 

Identifying  and  determining  objectives  is  primarily  met  through  user  inputs,  and 
not  as  a  result  of  process  computations.  Adherence  to  warfighting  doctrine  is 
accomplished  through  user  inputs  such  as  the  force  ratio  needed  for  ground  units  to 
attack,  or  the  strike  priorities  for  targets.  General  strategic  objectives  are  set  before  a  war 
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Figure  28.  Comparison  of  ISR  Purposes 


82 


begins  and  must  be  induced  by  the  user  via  the  input  files.  In  reality,  tactical  and 
operational  objectives  are  extremely  dynamic  and  stem  from  the  status  and  circumstances 
of  the  battle  learned  through  ISR  efforts.  In  THUNDER,  however,  these  efforts  are  not 
used  in  determining  objectives  for  operations. 

Identifying  enemy  centers  of  gravity  is  a  prime  focus  of  intelligence  activities. 
However,  THUNDER  primarily  uses  ISR  to  form  weapon/aircraft  assignments. 

THUNDER’S  ISR  does  aid  in  planning  which  targets  to  strike  in  the  air  war,  and 
can  also  help  an  aircraft  locating  a  target. 

Naval  aircraft  carriers  and  Tomahawk  Land  Attack  Missile  (TLAM)  platforms  are 
represented  in  THUNDER,  although  surface,  subsurface,  and  anti-submarine  warfare  is 
not  modeled.  Aircraft  on  the  carriers  conduct  operations  just  as  any  other  aircraft  based 
inland  along  with  TLAM  strikes. 

The  ground  war  is  highly  aggregated  in  THUNDER.  Unit  status  and  battle  results 
are  calculated  deterministically  through  the  ATCAL  process.  Ground  units  in  battle 
engagements  move  strictly  back  and  forth  along  pre-defined  sectors.  Therefore,  these 
units  do  not  decide  where  to  move,  since  they  always  want  to  move  forward.  It  is 
ultimately  a  question  of  ‘if  they  move,  and  which  way.  The  ISR  process  influences  the 
movement  of  units  in  combat  and,  therefore,  the  outcome  of  battles,  by  applying  a 
multiplier  to  the  lethality  and  target  availability  factors  used  in  ATCAL  equations  as 
discussed  previously.  This  multiplier  is  activated  whenever  a  standoff  reconnaissance 
aircraft  has  coverage  of  an  enemy  unit. 

THUNDER’S  ground  combat  involves  a  myriad  of  variables  including  unit 
strength,  supplies,  equipment,  lethality,  rate  of  fire,  shooter  bias,  flank  degrade,  target 
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priority,  and  unit  posture.  Since  the  ground  war  is  deterministic,  ISR  does  not  really  have 
an  impact  on  any  ground  unit  planning.  It  may  affect  second  echelon  or  reserve  type 
forces  as  they  may  move  along  a  network  to  carry  out  their  duties.  This  seems  to  be  a 
limited  representation  of  the  influence  of  intelligence  on  the  ground  war.  In  reality, 
ground  units  rely  heavily  on  intelligence.  Information  about  terrain,  enemy 
reinforcements,  second  echelon  forces,  enemy  supplies,  etc.  all  influence  operational  and 
tactical  decisions.  Ground  forces  base  decisions  of  when,  where,  and  how  to  attack  on 
the  information  they  receive  from  their  own  reconnaissance  equipment,  as  well  as  the 
intelligence  from  reconnaissance  aircraft  and  satellites.  However,  since  THUNDER’S 
ground  war  is  played  deterministically,  the  implementation  of  the  ISR  effect  through  a 
multiplier  seems  to  be  a  reasonable  compromise.  Although  ISR  does  not  aid  in  the 
planning  or  execution  of  operations  in  a  true  sense,  an  impact  on  battle  outcomes  can  be 
realized. 

It  is  interesting  to  note  that  although  the  user  can  script  observations  as  “Grid”  or 
“FLOT”  sensors  to  represent  ground  intelligence  collection,  these  observations  do  not 
impact  the  ground  war,  except  indirectly  through  air  targeting  of  the  enemy. 

Security  of  operations  and  countering  deception  and  surprise  usually  requires 
information  from  more  than  one  source  of  intelligence  since  nearly  every  type  of  sensor 
is  vulnerable  to  some  type  of  deception.  Fusing  and  comparing  various  sources  of 
information  allows  analysts  to  increase  their  confidence  in  what  they  evaluate  as  truth. 
THUNDER  does  not  compare  reports  from  different  sensors  in  its  “fusing”  process,  but 
simply  selects  the  report  from  the  sensor  that  has  the  highest  confidence  level  for  the 
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target.  To  obtain  a  reasonable  outcome,  the  user  must  carefully  assign  the  right 
confidence  levels  and  degradations. 

A  crucial  objective  of  intelligence  collection  is  to  prevent  or  counter  surprise 
attacks,  but  THUNDER  does  not  address  this  objective  at  all.  Knowing  where  troops  are 
massing  and  how  many  reinforcements  are  available  builds  knowledge  of  the  enemy  and 
aids  in  determining  enemy  intentions.  Again,  intelligence  does  not  affect  on-going 
operations  of  the  ground  war,  but  is  only  considered  after  the  fact  when  calculating 
losses.  A  possible  improvement  might  be  to  allow  a  unit  to  attack  with  a  lower  force 
ratio  if  it  is  believed  the  defender’s  reinforcements  are  delayed. 

Deceiving  the  enemy  and  the  element  of  surprise  are  highly  dependent  upon  the 
use  of  intelligence.  Knowing  the  command  and  control  systems  of  the  enemy  and  their 
intelligence  systems  facilitates  deception.  THUNDER’S  ATO  generator  can  target  these 
types  of  systems,  but  the  ability  to  plan  for  surprise  is  generally  lost.  However,  one  way 
to  accomplish  this  is  through  user  input.  For  example,  the  user  can  order  ground  units  to 
appear  at  a  location  at  a  designated  time,  but  this  really  has  no  correlation  with  the  status 
of  the  enemy’s  intelligence  capability.  Information  superiority  is  a  mission  that  is 
becoming  increasingly  important  in  today’s  world  of  technology.  The  ability  to  disable 
or  deceive  enemy  I  SR  capabilities  is  vital  to  accomplishing  this  mission.  THUNDER  has 
implemented  rule  language  that  allows  for  ISR  observations  to  be  lost  if  certain 
structures,  such  as  communication  centers,  are  destroyed.  Deception  is  hard  to  depict  in  a 
combat  model  because  of  the  nature  of  deception  activities.  Planning  for  deception  and 
ensuring  that  the  enemy  has  not  discovered  the  plan  does  not  fit  well  with  combat 
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modeling  quantitative  limitations.  The  “probability  of  coverage”  parameter  does  allow 
for  a  deception  effect  by  not  allowing  the  sensor  to  always  see  a  target  within  its  view. 

The  last  purpose  of  ISR  is  evaluating  effects  of  operations  and  re-orienting  forces 
or  terminating  operations.  Evaluating  effects  of  operations  is  a  BDA  function.  BDA  is 
usually  accomplished  by  more  than  one  sensor  because  of  the  complexity  of  assessing  the 
actual  damage.  BDA  starts  with  aircrew  debriefings  or  cockpit  video,  but  is  driven  by 
imagery.  In  THUNDER,  any  aircraft  defined  with  an  ISR  sensor  can  do  BDA.  This  can 
be  used  to  represent  the  aircrew  report  or  cockpit  video.  It  is  important  that  the  user 
configure  the  confidence  of  these  reports  appropriately,  as  generally  they  are  given  a 
lower  confidence  than  something  like  satellite  imagery. 

In  THUNDER,  the  nomination  of  a  target  for  re-strike  is  influenced,  through  a 
priority  multiplier,  by  whether  or  not  BDA  has  been  accomplished.  This  is  a  binary 
status,  and  does  not  take  into  account  the  confidence  of  a  BDA  report.  The  user  inputs 
the  perception  of  a  target  that  has  not  had  BDA  accomplished.  This  tells  THUNDER 
whether  to  consider  the  target  live  or  dead.  Targets  considered  dead  are  not  nominated 
for  re-strike.  Again,  only  one  sensor  has  to  accomplish  the  BDA  in  order  to  trigger  this 
perception,  regardless  of  the  confidence  of  that  report.  While  BDA  is  performed  on  three 
levels  -  physical,  functional,  and  system  damage,  THUNDER  has  incorporated  just  the 
physical  and  functional  aspects.  An  ISR  observation  will  report  target  characteristics, 
representing  physical  damage.  The  fact  that  ISR  observations  influence  whether  a  target 
is  considered  live  or  dead  can  be  seen  as  a  functional  aspect. 

The  usual  termination  criteria  in  THUNDER  is  a  stated  number  of  days  of  war 
completed  or  the  annihilation  of  one  side  which  leaves  no  targets  to  strike.  The  user  may 
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construct  “rules”  that  stop  aircraft  from  flying  when  a  percentage  of  certain  types  of 
targets  are  destroyed.  Therefore,  ISR  in  THUNDER  can  influence  the  termination  of 
operations  through  its  impact  on  air  targeting  and  the  ability  to  destroy  targets,  as  well  as 
the  perception  it  has  of  targets  (live  or  dead). 


6.3  Representation  of  ISR  P  rinciples 

As  previously  stated  in  Chapter  4,  there  are  numerous  principles  of  ISR  identified 
by  military  doctrine.  THUNDER  has  captured  some  of  the  principles,  while  some  are 
assumed.  Again,  assumptions  are  not  necessarily  inappropriate,  as  long  as  they  are 
known,  and  can  be  applied  validly.  Table  5  summarizes  the  ISR  principles-THUNDER 
relationship,  with  a  discussion  of  each  implemented  principle  following. 


Table  5.  ISR  Principles 


Principle 

Comments  on  THUNDER 

II 

Accuracy 

Allows  for  location  error  and  degrading 
confidence  levels 

E 

Timeliness 

Incurs  processing  time,  delays,  and 
probability  of  observation  being  lost 

Objectivity 

Assumed 

S 

Unity  of  Effort/Interoperability 

Cross-cueing  essence  in  locating  targets  and 
ground  force  multiplier 

Relevance 

Assumed 

Usability 

Assumed 

i 

Completeness 

Prioritization  of  targets  given  by  multipliers 

Readiness 

Not  modeled.  Demand-oriented  tasking 

S 

Fusion 

“Fuses”  various  sensor  information  by 
choosing  the  highest  confidence  level 

Accessibility 

Assumed 

Security 

Assumed 

■ 

Survivability  and  Sustainability 

Allows  for  redundancy  through  scripting 
(not  through  automated  scheduling).  Allows 
for  processing  delays  and  lost  observations 
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Accuracy.  This  is  implemented  in  THUNDER  through  the  use  of  confidence 
levels,  which  may  decrease  over  time.  Each  sensor  is  assigned  a  two-sigma  location  error 
applied  to  its  target  location  estimate.  An  additional  location  error  is  applied  at  the  time 
of  strike  based  on  the  age  of  the  report  used  for  initial  location. 

Timeliness.  Timeliness  of  ISR  reports  is  based  on  a  sensor’s  processing  time, 
which  may  be  a  random  variable.  Processing  delays  can  also  be  incurred  when  certain 
ISR  nodes  have  been  destroyed,  as  specified  by  user  input.  Also,  the  probability  of  an 
observation  being  “lost”  is  factored  in  when  ISR  nodes  are  destroyed  (subject  to  user 
input).  These  processing  delays  and  lost  observations  are  used  to  represent  a  variety  of 
occurrences  such  as  interrupted  or  lost  communications  or  analyst  oversight. 

Unity  of  Effort  /Interoperability.  The  fact  that  the  different  military  services, 
civilian  organizations,  and  allied  nations  have  interoperable  systems  and  are  unified  in 
effort  is  assumed  in  THUNDER.  One  element  in  THUNDER  where  this  has  been 
implemented  is  through  cross-cueing.  This  ability  is  demonstrated  in  THUNDER 
through  the  ability  to  “roll  the  dice”  again  if  an  AW  ACS  (or  J-STARS)  type  of  aircraft  is 
in  reach,  to  aid  in  target  location.  The  impact  of  cross-cueing  can  also  be  seen  by  the 
increase  in  lethality  and  target  availability  in  the  ground  war. 

Completeness.  It  is  assumed  that  the  “Commander”  has  all  available  and  relevant 
information  from  an  ISR  observation  to  accomplish  the  mission.  Prioritization  for 
obtaining  that  information  is  seen  through  the  prioritization  of  targets  for  intelligence 
collection.  Targets  are  prioritized  based  on  their  strike  priority,  which  is  then  multiplied 
by  various  ISR  factors,  such  as  a  BDA  multiplier. 
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Fusion.  THUNDER  “fuses”  observations  based  on  confidence  level.  This  issue 


was  discussed  in  Chapter  5. 

Survivability  and  Sustainability.  Mostly  seen  through  the  role  of  ISR  nodes.  As 
nodes  are  destroyed,  delay  time  and  probability  of  lost  observation  can  be  increased. 

Only  ISR  sensors  attached  to  aircraft  are  vulnerable  to  attack.  The  user  can  simulate  a 
satellite  being  “moved”  or  a  new  satellite  being  “launched”  during  the  course  of  the  war 
by  using  rules  that  set  the  probability  of  an  observation  being  lost  to  1 .00  until  the  sensor 
should  be  used. 

6.4  Types  of  Intelligence 

Intelligence  comes  in  different  forms.  In  THUNDER,  the  user  can  script 
HUMINT  and  OSINT  observations.  Because  any  aircraft  can  carry  a  sensor,  this  sensor 
can  be  considered  a  HUMINT  sensor  if  it  is  representing  an  aircrew  debriefing.  The  type 
of  intelligence  explicitly  accounted  for  in  THUNDER  is  Imagery  Intelligence.  Analysts 
can  “trick”  the  simulation  to  do  other  types,  but  it  is  laborious,  and  probably  not 
desirable.  THUNDER  6.6  will  be  able  to  model  SIGINT.  Since  HUMINT  and  OSINT 
are  scripted  inputs,  it  is  sufficient  to  explicitly  model  only  IMINT  and  SIGINT; 

MASINT  and  TECHINT  are  probably  not  necessary. 

6.5  Communications 

Communication  is  a  vital  part  of  all  military  operations,  including  the  ISR 
process.  SATCOM,  telephone,  radio,  fax,  electronic  message,  etc.  are  just  a  few  of  the 
media  to  transmit  information  such  as  requirements,  tasking  orders,  cross-cueing,  and  to 
permit  dissemination  of  ISR.  Communication  systems  and/or  C4I  systems  are  not 
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explicitly  modeled  in  THUNDER.  Some  communication  problems  and  their  effects  are 
captured  through  various  input  parameters.  For  instance,  a  processing  delay  is  incurred  if 
ISR  nodes  are  damaged  or  destroyed.  Processing  delays  may  decrease  if  an  aircraft  such 
as  an  AWACS  is  in  the  vicinity.  The  same  rules  can  apply  to  the  probability  of  an 
observation  being  lost.  An  observation  may  be  lost  for  a  variety  of  reasons  such  as  a 
message  being  jammed  or  garbled  or  through  saturation  of  a  communications  satellite. 

The  sensor’s  defined  processing  time  represents  the  time  it  takes  to  process  and  transmit 
the  data.  All  of  these  parameters  are  highly  aggregated  measures,  to  which  it  seems 
rather  difficult  to  assign  “accurate”  values.  It  may  be  desirable  to  explicitly  model 
communications  to  some  extent,  or  at  least  at  a  slightly  higher  resolution  than  currently 
employed. 

6.6  Initial  Preparation  of  the  Battlefield 

In  THUNDER,  the  Initial  Preparation  of  the  Battlefield  is  represented  by  a  data 
file  which  includes  the  time  since  the  last  observation  of  each  target  based  on  a  random 
draw  and  a  confidence  curve.  This  in  turn  translates  to  the  confidence  of  intelligence  data 
of  targets,  i.e.  usually  the  more  time  that  has  passed  since  that  last  observation,  the  lower 
the  confidence  level.  In  reality,  the  initial  assessment  entails  much  more  than  just  target 
status  and  contributes  overwhelmingly  to  the  timing  of  activities  and  courses  of  action 
considered  by  the  commander.  The  way  that  the  initial  assessment  must  really  be 
captured,  aside  from  the  data  file,  is  through  other  inputs  by  the  user.  Obviously  it  is  the 
user  that  decides  how  much  is  known  about  the  enemy  when  the  campaign  begins. 
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6.7  ISR  Process 


So  far  this  chapter  has  compared  the  ISR  purposes,  principles,  and  other  ISR 
components  in  reality  with  those  of  THUNDER.  The  following  section  will  compare  the 
ISR  process. 

6.7.1  User  Requirement 

Although  in  reality,  many  factors  drive  ISR  requirements,  e.g.  technological 
capabilities,  enemy  will,  allied  and  foreign  sentiments,  and  enemy  centers  of  gravity,  the 
ISR  requirements  in  THUNDER  are  basically  used  for  aircraft  targeting.  The  air 
planning  and  mission  generation  processes  within  THUNDER  attempt  to  model  AF 
doctrine  in  that  the  JFC  sets  intelligence  requirements,  which  the  JFACC  fulfills.  The  air 
planner  accomplishes  this  primarily  through  the  generation  of  target  lists,  which  are  then 
prioritized  for  reconnaissance/surveillance  missions.  The  user  can  impose  requirements 
through  scripting  or  inputting  planning  parameters  in  such  a  way  to  obtain  a  desired 
outcome.  For  example,  the  air  planner  does  not  generate  a  surveillance  mission  to  fulfill 
a  requirement  for  real-time  intelligence  to  aid  in  target  location,  but  the  user  can  either 
script  or  manipulate  certain  parameters  to  ensure  that  surveillance  of  targets  is 
accomplished,  thereby  modeling  the  desired  effect. 

Although  in  reality  ground  units  rely  on  intelligence  inputs  to  determine  their 
courses  of  action,  these  units  do  not  generate  requirements  for  intelligence  in 
THUNDER.  As  discussed  previously,  the  impact  of  intelligence  on  the  ground  war  is 
mainly  through  the  intelligence  benefits  to  aircraft  which  can  destroy  ground  units, 
equipment,  supplies,  etc.  Ground  units  do  receive  a  by-product  of  surveillance  missions 
in  that  having  surveillance  of  enemy  units  increases  the  lethality  or  target  availability; 
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however,  a  ground  unit  cannot  “request”  this  surveillance  in  THUNDER.  Yet  once 
again,  the  user  can  ensure  that  surveillance  takes  place  through  scripting  or  various  other 
inputs. 

Figure  29  summarizes  how  the  JFC  intelligence  requirements  of  each  of  the 
component  commanders  are  represented  in  THUNDER. 


THUNDER 


Primary  driver  of  ISR  requirements.  Employs 
reconnaissance  and  surveillance  aircraft  through 
ATO  generation.  The  user  scripts  orbital 
parameters  for  space  assets  that  make 
observations. 


{Benefits  from  air  and  space  reconnaissance  and 
surveillance,  but  does  not  generate  requirements. 
The  user  can  script  observations. 

{Special  Ops  not  really  played  in  THUNDER. 
HUMINT  is  scripted,  but  need  for  HUMINT 
does  not  drive  ISR  requirements. 

C  Naval  warfare  not  played  in  THUNDER.  The 
A.  user  can  script  observations  of  coastal  regions. 


Figure  29.  User  Requirement  Representations 


6.7.2  Planning  and  Tasking 

Figure  30  shows  the  Planning  and  Tasking  Process  in  THUNDER.  Figure  31 
shows  the  Planning  and  Tasking  Process  during  conflict. 

THUNDER  treats  planning  for  intelligence  collection  and  tasking  intelligence 
collectors  as  one  process.  Actual  planning  for  intelligence  collection  begins  by 
prioritizing  requests.  THUNDER  attempts  to  mimic  this  aspect  by  beginning  with  the 


92 


Assign  each 
standard 
target  a 
strike 
priority 


Apply  ISR 
multipliers: 
Target  Type 
Unnominated  Target 
BDA  Target 
Minimum  Priority 
Confidence  Level 


Figure  30.  Planning  and  Tasking  Intelligence  Collection  in  THUNDER 


standard  target  list  and  each  target’s  strike  priority.  Standard  targets  are  organized  into 
ISR  target  planning  classes  that  define  the  ISR  multipliers  to  apply  to  each  target’s  strike 
priority  and  therefore  prioritize  the  target  list  for  reconnaissance  and  surveillance 
(RECCE/SREC)  missions.  These  multipliers  allow  considerable  flexibility 


93 


Prioritize 

Targets 


Redundancy 

desired? 


Yes 


Consider 
weather, 
light  or 
geographical 
effects 


Consider 

vulnerability 

of 

platform/sensor 


Select 

collector(s) 


Possible 
dynamic  ' 
retasking 


for  the  user  in  planning  for  RECCE/SREC  missions.  Multipliers  are  applied  if  a  target  is 
not  nominated  for  strike  and  if  BDA  has  not  been  accomplished.  A  third  multiplier 
allows  the  user  to  establish  a  minimum  priority  that  the  target  may  have  for 
RECCE/SREC.  These,  once  again,  are  good  planning  tools  for  quantitative  items  such  as 
target  status.  Abstract  information  such  as  enemy  intentions  is  something  that  the  user 
must  somehow  manipulate  when  setting  up  initial  parameters. 

The  THUNDER  planning  module  lacks  coordination  between  air  and  space 
assets.  Air  assets  given  RECCE  or  SREC  missions  are  scheduled  through  the  ATO 
generator.  Space  assets  are  planned  for  automatically  through  their  specified  orbital 
parameters.  Of  course,  the  user  can  script  any  other  observations  desired.  The  planning 
process  in  THUNDER  shows  the  biggest  divergence  from  reality.  Figure  9  (page  39) 
shows  that  many  factors  are  considered  when  deciding  on  the  sensor/system  for 
collection.  In  THUNDER,  RECCE/SREC  mission  sorties  are  assigned  to  the 
aircraft/sensor  combination  that  has  the  highest  confidence  for  that  particular  target. 
Sensor  capability  in  regard  to  weather  and  range  to  target  are  also  considered.  Aircraft 
can  be  tasked  even  if  a  space  asset  can  make  a  collection,  and  vice  versa.  Although  this 
does  allow  for  possible  redundancy,  coordination  between  the  two  is  desired  to  allow  for 
maximum  coverage  of  the  greatest  number  of  targets.  In  reality,  all  types  of  intelligence 
collectors  (ground,  air,  space,  and  sea)  should  be  considered  when  planning  for 
collection. 

Another  factor  to  include  in  planning,  related  to  timeliness,  is  sensor/system 
availability.  The  ATO  generator  automatically  schedules  RECCE/SREC  aircraft.  When 
integrating  space  assets,  coverage  times  must  be  considered  since  the  times  that  a  space 
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asset  can  view  of  a  target  is  relatively  fixed.  Redundant  coverage  of  some  targets  may  be 
desirable,  especially  when  conducting  BDA.  THUNDER  does  not  schedule  missions 
specifically  based  on  the  need  for  a  redundant  observation,  but  it  does  not  prevent  it 
either  by  integrating  air,  space,  and  scripted  observations. 

Tasking  for  reconnaissance  aircraft  in  THUNDER  takes  place  in  the  ATO 
generator  at  the  same  time  as  the  planning.  Similarly,  satellite  tasking  is  assumed 
through  the  defined  orbital  parameters.  THUNDER  assumes  that  the  space  assets  are 
tasked  every  time  they  come  into  view,  and  perform  the  task  100%  of  the  time.  In  reality, 
some  sensors  may  not  always  be  able  to  gather  data  on  certain  targets  in  their  footprint 
due  to  various  factors  such  as  sun  interference  or  other  weather  effects.  Also,  it  is  not 
necessarily  a  “given”  that  just  because  a  satellite  is  overhead  that  it  will  be  tasked  to 
make  collections. 

Another  departure  from  reality  in  the  THUNDER  ISR  module  is  the  lack  of 
dynamic  re-tasking.  As  situations  change  and  new  information  becomes  available, 
collection  priorities  may  also  change.  In  THUNDER,  as  with  all  of  the  aircraft  sorties, 
dynamic  re-tasking  of  aircraft  is  not  possible.  This  is  the  same  for  all  collections.  Once 
the  planning  cycle  has  determined  the  RECCE/SREC  priorities,  these  will  not  change 
until  the  next  cycle. 

Finally,  tasking  aircraft  and  satellites  takes  time,  and  relies  upon  the  ability  to 
communicate.  Tasking  national  assets  requires  coordination  through  and  with  the 
appropriate  agency.  None  of  the  tasking  time  or  coordination  is  represented  in 
THUNDER,  and  probably  isn’t  necessary.  The  usual  Planning  Cycle  in  THUNDER  is  12 
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hours,  where  in  reality  it  is  48-72  hours.  This  extra  time  should  ensure  that  at  12  hours 
out,  the  tasking  has  been  ordered. 

6.7.3  Collection 

Collection  is  scheduled  by  the  ATO  generator  for  RECCE  and  SREC  missions. 
Collection  times  by  satellites  are  given  by  the  orbital  parameters  of  the  satellite/overhead. 
Any  aircraft  equipped  with  a  sensor  may  make  observations  after  a  strike.  Sensors 
defined  in  THUNDER  as  “Grid”  or  “FLOT”  sensors  can  be  used  to  represent  ground- 
based  collection.  The  user  must  script  all  other  types  of  collection.  Scripting  permits  the 
user  to  define  almost  any  type  of  collector.  One  limitation  of  THUNDER  is  that  aircraft 
can  only  carry  one  sensor.  In  reality  this  is  not  the  case.  For  example,  a  high  altitude 
UAV,  such  as  Global  Hawk,  can  carry  multiple  sensors  such  as  electro-optical,  infra-red, 
and  synthetic  aperture  radar. 

Since  THUNDER  treats  carriers  as  floating  airbases,  any  naval  aircraft  that 
performs  surveillance  or  reconnaissance  missions  can  be  modeled  and  scheduled  just  as 
any  other  land-based  aircraft.  However,  intelligence  of  enemy  naval  assets  normally 
gathered  by  naval  platforms  will  not  be  captured  in  THUNDER.  This  has  little  impact  on 
the  theater  level  warfighting  envisioned  by  THUNDER  since  it  does  not  model  open 
ocean,  ship  vs.  ship  warfare. 

Army  intelligence,  or  any  ground-based  intelligence,  is  not  modeled  explicitly  in 
THUNDER.  The  user  can  mimic  these  type  of  collectors  by  using  “Grid”  or  “FLOT” 
sensors.  In  reality,  Army  intelligence  gathering  capabilities  depend  to  a  large  extent  on 
the  role  of  the  unit,  its  proximity  to  the  FLOT,  and  its  status,  (i.e.,  engaged,  recuperating, 
etc.)  Numerous  types  of  equipment  can  gather  intelligence,  whether  purposely  or  as  an 


97 


important  by-product  of  normal  activities.  These  collections,  in  turn,  aid  in  Army 
operations  at  both  the  tactical  and  operational  level. 

6.7.4  Analyze 

Processing  time  is  captured  through  random  delays  and  user  input.  This 
also  includes  analysis  time.  Analyst  error  or  oversight  can  be  simulated  through  the 
probability  that  an  observation  is  lost.  Again,  determining  the  number  to  be  used  for 
analysis  time  and  probability  of  error  is  not  an  easy  task.  When  fusing  data  from  many 
sources,  THUNDER  uses  the  highest  confidence  level  for  each  target;  the  confidence 
level  considers  the  time  since  the  observation  was  made.  This  “fusion”  process  is  fairly 
realistic  as  long  as  the  sensor  confidence  levels  truly  reflect  the  ability  of  the  sensor. 
However,  as  with  any  model,  THUNDER  does  not  capture  the  human  judgment  often 
involved  when  human  analysts  face  “special  cases”  and  their  experience  and  knowledge 
drive  their  actual  decisions.  For  example,  it  is  possible  that  specifications  of  a  target 
found  in  open  source  material  may  be  more  useful  than  a  report  gathered  by  ISR  assets, 
even  though  open  source  is  generally  considered  less  reliable.  The  integration  of  old  data 
with  new  data  to  make  better  assessments  is  also  lost  in  THUNDER.  Old  data  is  simply 
replaced  by  new  observations.  Also,  since  no  other  observation  is  considered,  any 
dichotomy  between  sensors  will  not  be  realized.  For  example,  if  one  sensor  reports  10 
tanks  with  80%  confidence,  and  another  sensor  reports  3  tanks  with  75%  confidence, 
only  the  10  tanks  will  be  considered,  whereas  in  reality,  the  knowledge  of  another  highly- 
confident  sensor  that  produces  a  significantly  different  report  may  invoke  the  need  to  re¬ 
observe  the  target. 
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6.7.5  Dissemination 


Dissemination  in  THUNDER  can  be  represented  by  the  sensor’s  processing  time. 
Dissemination  can  be  stopped  through  the  “probability  of  lost  observation”  parameter. 

6.7.6  Evaluate 

In  THUNDER,  all  observations  are  assumed  to  be  relevant  until  an  observation 
with  higher  confidence  is  reported,  then  the  old  observation  is  discarded.  The  end  user  of 
the  product  is  assumed  to  be  satisfied.  The  feedback  loop  is  not  modeled. 

6.7.7  Apply 

The  ISR  observations  are  applied  by  updating  air-to-ground  target  statuses.  Also, 
it  is  applied  in  its  influence  on  the  ground  war. 

6.8  Summary 

This  chapter  has  presented  a  comparison  of  the  ISR  purposes,  principles,  and 
processes  involved  in  the  real  world  and  in  THUNDER.  Overall,  THUNDER  has 
captured  most  of  the  critical  elements  of  the  ISR  process.  Some  assumptions  that 
THUNDER  makes  in  regard  to  ISR  are: 

-  ISR  and  C4  systems  within  military  services,  among  military  services,  among 
military  and  civilian  organizations,  and  among  nationalities  are  all 
interoperable. 

-  Observations  are  always  analyzed  objectively,  and  the  same  confidence  level 
is  always  assigned  to  a  particular  sensor 

-  The  observation  from  the  sensor  with  the  highest  confidence  level  is  always 
considered  the  best. 

-  Satellites  always  make  observations  whenever  the  theater  is  in  view. 

-  Observations  always  fulfill  the  user’s  request. 
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-  Only  sensors  attached  to  aircraft  are  vulnerable  to  attack. 

Numerous  parameters  and  algorithms  are  used  within  THUNDER  to  represent 
ISR.  Many  THUNDER  ISR  elements  mimic  the  real  world,  or  at  least  mimic  the  effects, 
in  good  fashion.  Some  of  the  better  qualities  of  THUNDER’S  ISR  module  include: 

-  ISR  sensors  can  be  distinguished  through  capability  and  confidence  level 
allowing  for  flexibility  and  ability  to  be  adapted  to  a  wide  variety  of  scenarios. 

-  ISR  sensors  are  modeled  as  real  or  batch;  each  has  a  processing  time  that  can 
be  adversely  affected  by  the  destruction  of  ISR  nodes. 

-  Aging  of  ISR  observations  is  taken  into  account  when  considering  target 
nomination,  confidence  of  the  report,  and  target  location  error. 

-  ISR  observations  affect  prioritization  of  targets. 

-  THUNDER  makes  an  attempt  to  “fuse”  ISR  observations. 

There  are  also  some  limitations  to  THUNDER’S  ISR  module.  Many  of  these 
limitations  come  in  the  interaction  between  ISR  assets  and  the  ground  war.  This  is  due  in 
part  to  the  fact  that  the  ground  war  is  played  deterministically,  limiting  the  ability  of  ISR 
to  affect  ground  war  decisions.  Some  of  the  most  significant  disadvantages  of 
THUNDER’S  ISR  module  are: 

-  The  ground  war  has  no  impact  on  ISR  scheduling.  Support  of  the  ground  war 
does  not  factor  into  RECCE/SREC  mission  scheduling. 

-  ISR  has  no  impact  on  ground  units’  courses  of  action.  Knowledge  of  enemy 
reserves  or  status  of  supplies  is  not  considered. 

-  Grid,  FLOT  and  Space  sensors,  as  well  as  RECCE  missions,  do  not  aid  the 
ground  war. 

-  ISR  collection  is  not  coordinated  among  satellites,  aircraft,  and  scripted 
events,  resulting  in  less  than  the  maximum  number  of  targets  being  observed. 
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-  In  the  target  nomination  rules,  BDA  is  considered  to  be  accomplished 
regardless  of  the  confidence  level  of  the  sensor. 

-  Aircraft  can  only  carry  one  ISR  sensor. 

-  Dynamic  re-tasking  of  ISR  assets  is  not  possible. 

Because  THUNDER  is  a  campaign-level  model,  some  parameters  and  process 
elements  are  aggregated  to  represent  numerous  possible  effects.  Some  of  the  elements 
aggregated  in  THUNDER’S  ISR  module  are: 

-  Processing  Time  <=  Time  to  gather  the  intelligence,  process  it,  relay  it  to  the 
analyst,  analyze  the  data,  and  disseminate  the  information  to  the  user  is 
aggregated  into  a  randomized  processing  time. 

-  Processing  Delay  and  Probability  of  Lost  Observation  <=  Problems  such  as 
interrupted  communications,  analyst  errors,  and  the  effect  of  information 
overflow  Eire  represented  by  a  delay  added  to  the  processing  time  and  a 
probability  that  the  observation  is  lost  and  not  used. 

-  Targeting  Decisions  <=  All  user  requirements  are  aggregated  into  decisions 
made  about  targeting. 

-  Air  Planning  Module  <=  The  Planning  and  Tasking  processes  for 
SREC/RECCE  missions,  and  their  coordination  among  military  and  civilian 
organizations,  takes  place  when  the  ATO  generator  schedules  the  missions. 

-  Lethality  and  Target  Availability  <=  All  ISR  effects  on  ground  units  are  seen 
only  in  the  lethality  and  target  availability  factors  used  for  attrition. 

Although  many  ISR  elements  are  represented  in  THUNDER,  many  of  these  rely 
on  user  input.  Therefore,  the  user’s  knowledge  and  ability  to  input  and  script  ISR 
parameters  and  elements  correctly  is  critical  to  the  process,  and  in  effect,  to  the  modeling 
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capability  of  THUNDER  itself.  Some  of  the  effects  that  rely  upon  user  input  or  scripting 
are: 

-  Overall  knowledge  of  the  situation  and  enemy  at  the  onset  for  air 
apportionment  and  target  priority  decisions 

-  Sensor  capabilities  -  range,  accuracy,  confidence,  perception  of  live  targets  as 
live  and  live  targets  as  dead 

-  Impact  of  BD  A  on  targeting  decisions 

-  Maximum  force  multiplier  of  SREC  on  ground  war 

-  Influence  on  processing  time  due  to  ISR  nodes  being  destroyed 

-  Representation  of  ground-based  and  sea-based  sensors  through  the  scripting  of 
Grid  and  FLOT  sensor  events 

-  Scripting  of  units  to  appear  to  simulate  a  surprise  attack 

-  Launching  or  moving  of  satellites  during  the  course  of  the  war 
Many  of  the  real  world  ISR  elements  and  effects  are  captured  through 

THUNDER’S  automated  processes  or  through  user  scripting.  The  next  chapter  will 
verify  whether  THUNDER’S  processes  actually  show  ISR  effects  in  battle  outcomes. 
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7.  Sensitivity  Analysis  of  THUNDER  ISR  Module 


7.1  Purpose 

In  order  to  evaluate  the  effectiveness  of  THUNDER’S  ISR  implementation,  an 
experiment  examining  the  sensitivity  of  the  quality,  quantity,  and  timeliness  of  ISR 
observations  on  battle  outcomes  was  performed.  The  purpose  of  the  experiment  was  to 
verify  that  THUNDER  is  sensitive  to  ISR  changes  so  it  can  validly  be  used  for 
comparative  analysis  of  competing  ISR  systems.  For  example,  when  trying  to  compare 
two  systems,  and  one  guarantees  “faster”  intelligence  data,  it  must  be  established  that 
THUNDER  is  sensitive  to  that  difference  in  order  to  show  the  benefit  of  more  timely 
intelligence.  This  experiment  does  not  attempt  to  “quantify”  the  value  of  intelligence  by 
concluding,  for  example,  that  if  you  have  faster  intelligence,  you  kill  1000  tanks  versus 
700  tanks,  but  it  is  merely  interested  in  the  fact  that  more  tanks  were  killed.  It  is  only 
these  comparisons  and  sensitivities  among  ISR  capabilities  that  are  evaluated. 

7.2  Scenario 

A  30-day  war  using  THUNDER’S  Middle  East  (ME)  database  was  used  for  this 
experiment.  The  ME  database  is  an  unclassified  scenario  distributed  with  THUNDER  in 
which  the  battlefield  is  oriented  to  resemble  a  Desert  Storm  scenario.  One  of  the 
problems  with  this  scenario,  however,  was  the  use  of  the  ISR  module.  The  scenario 
defaults  to  using  “High”  resolution  for  ISR.  Many  of  the  sensors  defined  in  the  scenario 
are  never  used,  and  only  one  reconnaissance  aircraft,  RF-4,  is  defined  for  the  Blue  side. 
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Because  of  the  need  for  more  reconnaissance  aircraft,  one  squadron  each  of  High- 


Altitude  Endurance  UAVs,  U-2s,  and  RC-135s  was  added.  The  specifications  for  the 
aircraft  were  taken  from  unclassified,  open  literature  sources  [22, 28, 41, 48].  Absolute 
accuracy  of  system  specifications  was  not  as  important  as  simply  having  more  sensors 
available  in  the  scenario. 

The  Blue  side  in  the  ME  database  is  superior  to  the  Red  side.  To  avoid  biasing 
battle  outcomes  due  to  this  superiority,  Blue  capability  was  reduced  and  Red  capability 
was  increased.  Some  Blue  squadrons  were  removed  and  all  other  Blue  squadrons  were 
reduced  by  20-25%.  In  order  to  increase  Red’s  ability  and  desire  to  push  forward,  the 
tactical  march  rate  for  the  Red  side  was  increased  by  1000  and  the  objective  of  the  Red 
side  was  increased  from  0  to  1,000,000  meters.  Finally,  40%  of  the  Blue  squadrons  and 
65%  of  Blue  ground  units  were  delayed  from  arriving  at  the  theater  anywhere  from  1  to  7 
days  after  combat  had  begun.  Table  6  summarizes  the  modifications  made  to  the  ME 
database. 


Table  6.  Changes  to  ME  database 


ME  Database 

Modified  Database 

Number  of  Blue  squadrons 

51 

48 

Number  of  Blue  aircraft 

1144 

724 

Number  of  Blue  types  of  aircraft 

19 

22 

Number  of  Red  squadrons 

19 

19 

Number  of  Red  aircraft 

l 

454 

454 

Red  tactical  march  rate 

+1000  for  each  unit  type 

Red  objective 

0 

1,000,000 

Blue  unit/squadron  orders 

None 

Some  arrive  1-7  days  late 
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7.3  Design  of  Experiment 

The  design  of  the  experiment  was  a  face-centered  central  composite  design 
(CCD)  with  3  factors  -  quality,  quantity,  and  timeliness.  Central  composite  designs  are 
widely  used  for  second-order  models.  The  CCD  has  three  components:  factorial  points, 
axial  points,  and  center  runs. 

The  factorial  points  represent  a  variance  optimal  design  for  a  first- 
order  model  or  a  first-order  +  two-factor  interaction  type  model.  Center 
runs  clearly  provide  information  about  the  existence  of  curvature  in  the 
system.  If  curvature  is  found  in  the  system,  the  addition  of  axial  points 
allows  for  efficient  estimation  of  the  pure  quadratic  terms  [36:298]. 

A  graphical  depiction  of  the  face-centered  CCD  is  shown  in  Figure  32. 
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Figure  32.  Face-centered  Central  Composite  Design 


The  face-centered  CCD  was  chosen  so  that  the  extreme  values  can  be  measured 
and  a  response  surface  can  be  generated  within  the  bounds  of  the  design  space.  It  is 
desirable  to  include  points  that  are  at  the  extremes  as  this  results  in  the  most  attractive 
scaled  prediction  variance  [36:3 13].  The  prediction  variance  will  change  depending  on 


105 


the  position  in  the  design  space,  and  it  reflects  how  well  the  model  can  predict  the 
response.  The  traditional  CCD  could  not  be  accomplished  because  the  axial  point  design 
settings  were  infeasible. 


As  can  be  seen  in  Figure  32,  the  region  defined  by  the  face-centered  CCD  is 
cuboidal.  The  design  is  not  rotatable,  which  would  provide  for  a  constant  scaled 


prediction  variance  for  any  two  points  the  same  distance  from  the  design  center  [36:306]. 

However,  according  to  Meyers  and  Montgomery 

. .  .rotatability  or  near-rotatability  is  not  an  important  priority  when  the 
region  of  interest  is  clearly  cuboidal.  Rotatability  (or  near-rotatability) 
is  a  useful  option  that  comes  from  spherical  or  near-spherical  designs; 
these  designs  are  certainly  appropriate  for  spherical  regions  of  interest 
or  regions  of  operability,  and  they  are  less  appropriate  with  cuboidal 
regions  [36:313]. 

The  design  matrix  for  the  face-centered  CCD  is  shown  in  Equation  (3),  where 
each  row  in  the  transposed  matrix  denotes  a  factor.  The  top  row  represents  timeliness, 


the  second  row  is  quantity,  and  the  bottom  row  signifies  quality.  A  setting  of  -1 


represents  the  “Low”  setting,  1  represents  the  “High”  setting,  and  0  represents  the  center 
point. 


Response  = 


-11-11- 

1  1  -1 

1 

0 

0 

0 

0 

-1 

1 

0 

0 

0 

0 

0 

0 

1 

-1-11  1  - 

1  -1  1 

1 

0 

0 

-1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

-1  -1  -1  -1 

1  1  1 

1 

-1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0. 

(3) 


Note  that  the  matrix  includes  6  center  point  replications  (seen  in  the  last  6 
columns  of  (3)  above).  For  cuboidal  designs,  two  center  runs  will  suffice  to  stabilize  the 
scaled  prediction  variance  [36:313].  However,  doing  additional  center  runs  allows  for 
more  degrees  of  freedom  when  estimating  the  error  [36:1 13]. 
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Since  THUNDER  is  a  stochastic  model,  30  replications  were  performed  at  the 
first  eight  design  points,  which  represent  the  full-factorial  portion  of  the  design,  and  20 
replications  were  performed  at  the  six  axial  design  points.  For  the  center  runs,  4 
replications  at  each  center  point  in  the  design  matrix  were  completed  for  a  total  of  24 
replications  at  the  center  point.  The  mean  response  over  all  replications  was  calculated, 
and  this  mean  was  used  in  the  response  vector  for  the  design  matrix. 

Every  run  was  independent.  No  attempt  was  made  to  correlate  the  runs  at  the 
various  design  points  due  to  the  large  number  of  stochastic  processes  in  THUNDER. 
Synchronization  of  random  numbers  in  THUNDER  is  difficult,  if  not  impossible,  to 
achieve.  An  attempt  to  use  common  random  numbers  may  correlate  and  synchronize 
processes  at  the  start  of  a  run,  but  this  would  soon  diverge  and  synchronization  would 
cease.  Banks,  Carson,  and  Nelson  suggest  that  “If  synchronization  is  not  possible. . .,  use 
independent  streams  of  random  numbers...”  [3:483]. 

7.4  Parameters 

Table  7  summarizes  the  parameter  settings  used  corresponding  to  the  face- 
centered  CCD  matrix  of  Equation  (3).  Each  factor  -  quantity,  quality,  and  timeliness  - 
contains  parameters  within  THUNDER  varied  together  for  each  setting  of  low,  center, 
and  high.  Only  the  Blue  side’s  parameters  were  changed  for  the  experiment,  Red 
capabilities  were  left  untouched. 
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Table  7.  Parameter  Settings  for  each  Factor  Level 


Factor 

Parameters 

Low 

Center 

High 

Quality 

Perception  (Live  as  Live) 

0.25 

0.5 

0.75 

Two-Sigma  Target  Location  Error 

5000 

2550 

100 

Ground  War  Multiplier  (Lethality  and  Availability) 

1.5 

2.25 

3.0 

SREC  Air-to-Ground  Update 

.25 

.5 

.75 

Timeliness 

Processing  Time 

18 

13.5 

9 

Quantity 

Probability  In  Coverage 

0.25 

0.5 

0.75 

Probability  of  Initial  Coverage 

0.25 

0.5 

0.75 

Probability  of  Lost  Observation 

0.75 

0.5 

0.25 

The  “Low”  setting  represents  the  study’s  worst  ISR  capability,  with  “High”  being 
the  best.  “Center”  represents  the  center  point.  Some  preliminary  results  from  an  AFSAA 
study  [52]  indicate  little  difference  in  some  battle  outcomes  between  the  absolute 
extremes  (such  as  0  and  1.0)  for  the  above  parameters.  Therefore,  0.25  and  0.75  are  used 
as  the  extremes.  The  Two-Sigma  Target  Location  Error  setting  was  used  in  the  AFSAA 
study  as  was  the  Processing  Time.  The  Processing  Time  was  set  at  9  and  18  so  that  the 
effect  wouldn’t  just  be  to  shift  the  effects  to  the  next  Air  Planning  Cycle  (the  usual  Air 
Planning  Cycle  in  THUNDER  is  12  hours).  By  choosing  9  as  the  “High”  setting,  any 
observation  taken  before  hour  3  of  the  Cycle  is  available  for  planning  the  next  Cycle. 
Anything  taken  after  hour  3  is  not  available  until  the  next  Cycle  has  already  been  planned 
(Figure  33a).  By  choosing  18  as  the  “Low”  setting,  nothing  is  available  for  the  next 
Planning  Cycle,  but  if  the  observation  is  made  before  hour  6,  it  is  available  to  plan  for  the 
third  Planning  Cycle.  Anything  after  hour  6  is  not  available  until  the  fourth  Planning 
Cycle  (Figure  33b).  Choosing  these  settings  adds  randomness  in  that  the  availability  may 


108 


be  similar  at  both  settings,  or  it  may  be  two  Cycles  apart,  but  it  will  not  just  shift  the 
planning  over  a  cycle. 


Observation  _ ►  Used  for  Planning 


Observation  —  ►  Used  for  Planning 

1  1 


Planning  Cycle  1  Planning  Cycle  3 

Planning  Cycle2  Planning  Cycle  4 


a)  Processing  Time  set  to  9 


Observation - ^  Used  for  Planning 


Observation- 


-►  Used  for  Planning 

1 


Planning  Cycle  1  Planning  Cycle  3 

Planning  Cycle2  Planning  Cycle  4 

b)  Processing  Time  set  to  18 


Figure  33.  Processing  Time  Parameters  Impact  on  Air  Planning  Cycle 
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7.5  Measures  of  Outcome  (MOO)/Measures  of  Effectiveness  (MOE) 

Five  MOO/MOEs  were  chosen  corresponding  to  battle  outcomes  expected  to 

demonstrate  sensitivity  to  ISR  capabilities.  These  measures  are: 

Total  Red  ground  equipment  killed.  Equipment  includes  tanks,  infantry 
vehicles,  armored  personnel  carriers,  trucks,  and  artillery. 

-  Red  equipment  killed  in  ground  battle. 

-  Red  equipment  killed  by  air  missions. 

-  Air  Loss  Exchange  Ratio:  Ratio  of  Red  air  losses  to  Blue  air  losses.  Air 
Losses  include  air-to-air  losses,  surface-to-air  losses,  and  aircraft  lost  on  the 
ground. 

-  Percentage  of  Red  strategic  targets  killed.  Strategic  targets  include  Nuclear, 
Biological,  Chemical  (NBC)  facilities,  communication  centers,  command 
bunkers,  refineries,  power  plants,  and  air  defense  radars. 

7.6  Statistical  Analysis  Methods 

To  illustrate  the  responsiveness  of  THUNDER  to  ISR  input  parameters,  a 
combination  of  statistical  methods  was  used  based  upon  least  squares  linear  regression 
and  determining  the  difference  between  two  means. 

7.6.1  Least  Squares  Linear  Regression 

Linear  regression  involves  using  independent  predictor  variables  to  estimate  a 
model,  or  function,  that  can  be  used  to  predict  a  response.  A  first-order  model  is 
represented  by  the  following  equation: 

y  =  p0  +  PiXi  +  P2X2  +  ...  +  PkXk  +  s  (4) 

where  y  is  the  response  variable,  Xj  are  the  independent  variables  (or  factors),  pi  are  the 
regression  coefficients  which  represent  the  expected  change  in  response  y  per  unit  change 
in  Xj  when  all  remaining  independent  variables  Xj  (i^j)  are  held  constant,  s  is  the  error 
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term  which  is  assumed  to  be  independent  and  identically  normally  distributed  with  a 
mean  of  zero  and  constant  variance. 

The  term  “linear”  regression  is  used  because  the  model  is  linear  in  the  parameters 
of  pi.  Often,  first  order  models  (first  order  referring  to  the  independent  x  variables)  are 
not  adequate  in  representing  the  model  relationships.  If  curvature  exists  or  interaction 
terms  are  significant,  second  order  models  must  be  used.  This  study  uses  second  order 
models  to  fit  the  responses.  An  example  of  a  second  order  model  is  shown  below: 

y  =  po  +  P1X1  +  P2X2  +  PnX]2  +  P22X22  +  P12X1X2  +  e  (5) 

The  method  of  least  squares  calculates  the  p’s  such  that  the  sum  of  the  squares  of 
the  errors,  s’s,  are  minimized.  An  error  is  the  difference  between  the  observed  value  and 
the  corresponding  value  predicted  by  the  fitted  model. 

7.6.2  Testing  for  Significance  of  Individual  Regression  Coefficients 

Variables  that  are  not  important  to  the  model  can  cause  the  mean  square  error  to 
increase  and  therefore  decrease  the  usefulness  of  the  model  [36:31],  To  ensure  that  only 
significant  variables  are  included,  a  t-test  is  used.  The  null  hypothesis  is  Ho:  Pj  =  0,  with 
the  alternative  of  Ha:  Pj  *  0.  If  the  null  hypothesis  is  not  rejected,  the  coefficient,  Pj,  for 
the  Xj  variable  is  not  significant,  and  Xj  is  removed  from  the  model.  The  test  statistic  is 
given  by 


(6) 
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where  bj  is  the  estimate  of  ft,  and  Qj  is  the  diagonal  element  of  (X’X)'1  corresponding  to 
bj,  where  X  represents  the  matrix  of  independent  x  variables.  The  null  hypothesis  is 
rejected  if  |to|  >  ta/2,n-k-i»  where  n  is  sample  size  and  k  is  the  number  of  independent 
variables.  The  t-test  was  used  in  this  study  during  the  “screening”  phase  to  determine 
which  variables  to  include  in  the  model.  A  p-value  was  computed  to  determine  whether 
or  not  to  accept  or  reject  the  null  hypothesis.  A  p-value  is  the  smallest  value  of  a  for 
which  the  null  hypothesis  can  be  rejected  [35:432].  For  this  study,  a  variable  with  a  p- 
value  of  greater  than  0.10  was  considered  insignificant. 

7,6.3  Testing  for  Significance  of  Regression 

Testing  for  significance  of  regression  determines  whether  any  of  the  Pj 
coefficients  are  significant.  The  null  hypothesis  is  Ho:  Pi  =  P2  =  •  •  •  =  Pk  =  0,  with  an 
alternative  of  Ha:  Not  all  Pi  equal  zero.  Rejecting  the  null  hypothesis  implies  that  at  least 
one  of  the  independent  variables  is  significant  to  the  model.  The  test  statistic  is 

p  —  SSrJJc  _  MSr 
0  ~  SSE/(n-k-\)  MSe 


where  SSr  is  the  sum  of  squares  due  to  the  model  (or  regression)  and  SSg  is  the  sum  of 
squares  due  to  the  residual  (or  error).  The  formulas  for  these  estimates  are 


55, 


b'X'y 


Art  \ 

2>. 

f= 1 

n 


(8) 


SSE=y'y-b'X'y 


(9) 
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H0  is  rejected  if  F0  >  Fa>  k,  n-k-i-  A  p- value  is  computed  for  this  test  statistic  to 
determine  whether  or  not  the  model  is  significant.  In  this  study,  the  model  was 
considered  significant  if  the  p-value  is  less  than  0.05. 

The  coefficient  of  multiple  determination,  R2,  is  another  estimate  that  implies  the 
appropriateness  of  the  model.  R2  is  the  measure  of  the  amount  of  reduction  in  the 
variability  of  the  response  obtained  from  the  independent  variables  used  [36:30], 


R2  = — —  (10) 

SSR  +  SSE 

However,  adding  an  independent  variable  will  always  increase  R2,  so  a  better  measure  is 
given  by  an  adjusted  R2,  which  does  not  always  increase  with  the  addition  of  a  variable. 


r2  SSE/(n-p) 

“*  (SS.+SS, )/(»-!) 


(11) 


where  p  is  the  number  of  Pi’s.  Higher  R2adj  values  indicate  a  better  fit  to  the  data. 

To  verify  that  model  assumptions  are  not  violated,  residual  analysis  is  performed. 
This  involves  verifying  that  the  error  terms,  £j’s,  are  independent  and  identically  normally 
distributed  with  mean  zero  and  constant  variance.  For  this  study,  a  scatter  plot  of  the 
residuals  against  the  predicted  vales  was  used  to  visually  verify  independence  and 
constant  variance.  The  Shapiro  Wilk  test  was  used  to  verify  normality.  The  null 
hypothesis  of  the  test  is  that  the  distribution  is  normal.  For  this  study,  residuals  with  a  p- 
value  of  less  than  0.05  were  considered  non-normal. 
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7.6.4  Testing  for  Significant  D  ifferences  in  Mean/Median  Response 

Ninety-percent  confidence  intervals  were  constructed  between  each  of  the  first 
eight  design  points  to  test  for  differences  in  mean/median  response.  For  design  points 
that  had  normally  distributed  data,  90%  two-sample-t  confidence  intervals,  assuming 
unequal  variances,  were  constructed.  Law  and  Kelton  [33:319]  state  that  constructing 
confidence  intervals  to  test  for  differences  in  mean  response  is  preferable  to  testing  the 


hypothesis  that  the  mean  responses  are  equal  for  the  following  two  reasons: 

1 .  Since  the  model  is  only  an  approximation  to  the  system,  (the 
hypothesis)  will  clearly  be  false  in  almost  all  cases  [33:319]. 

2.  A  confidence  interval  provides  more  information  than  the 
corresponding  hypothesis  test.  If  the  hypothesis  test  indicates 
that  (the  means  are  not  equal),  then  the  confidence  interval  will 
provide  this  information  and  also  give  an  indication  of  the 
magnitude  by  which  (the  mean  of  the  first  system)  differs  from 
(the  mean  of  the  second  system)  [33:319]. 

Because  the  replications  between  design  points  were  independent,  and  equal  variances 
could  not  be  assumed,  the  two-sample-t  confidence  interval  first  introduced  by  Welch  in 
1938  was  used  [33:589].  The  confidence  intervals  were  formed  in  the  following  manner. 
Let  Xi  and  X2  be  the  average  of  the  responses  for  two  design  points,  and  Xy  and  X2j  be 
the  j*  response  in  the  corresponding  design  point.  Calculate  the  following  equations: 


30 


X ,  =  £  x , 

j= 1 


(12) 


for  i=l,2.  Degrees  of  freedom  are  estimated  as: 


(13) 
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are: 

1 .  Both  samples  are  random  samples  from  their  respective  populations 
[6:216]. 

2.  In  addition  to  independence  within  each  sample,  there  is  mutual 
independence  between  the  two  samples  [6:216]. 

3.  The  measurement  scale  is  at  least  ordinal  [6:216]. 

The  procedure  for  calculating  a  90%  confidence  interval  for  difference  in 
the  median  is  as  follows.  Obtain  ordered  values,  <  . . .  <  U*mn^  of  Xu  -  X2j  for 
i=l . .  .m  and  j=l . . . .n,  with  m  and  n  being  the  sample  sizes.  The  lower  bound  for 
the  90%  confidence  interval  is  U(C),  and  the  upper  bound  is  u(mn+1'C).  For  a  90% 
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confidence  interval,  C  can  be  approximated  [25:79]  by 


Cs 


mn 

~2 


mn(m  +  n  + 1) 
12 


(16) 


A  significant  difference  in  median  response  between  design  points  is  concluded  if 
the  confidence  interval  does  not  contain  zero. 


7.7  Results 

For  each  MOO/MOE,  a  screening  experiment  was  performed  to  determine 
significant  variables  to  include  in  the  model.  The  t-test  for  significant  coefficents  was 
used,  as  well  as  a  graphical  look  at  the  influence  of  quality,  quantity,  and  timeliness.  The 
model  was  then  constructed  and  verified  with  the  F-test  for  linear  significance.  All 
models  proposed  in  this  study  passed  the  model  assumptions  for  the  residuals.  The 
model  yields  a  response  function  that  was  used  to  form  a  surface  plot.  For  each 
MOO/MOE,  three  surfaces  plots  are  shown,  each  corresponding  to  timeliness  set  at  either 
the  low,  center,  or  high  level;  Air  Loss  Exchange  Ratio  has  quality  at  the  various  settings. 
Each  surface  has  quantity  and  quality  plotted  on  the  two  horizontal  axes,  with  the 
response  of  the  MOO/MOE  on  the  z-axis;  Air  Loss  Exchange  Ratio  has  quantity  and 
timeliness  plotted  on  the  axes.  Twenty-five  coordinates,  corresponding  to  quantity  and 
quality  set  at  -1,  -0.5, 0,  .5,  and  1  were  used  to  construct  the  surface  plots;  the  rest  of  the 
points  were  interpolated  by  Mathcad  software.  Finally,  a  table  indicating  differences  in 
mean/median  response  among  the  first  eight  design  points  is  shown.  The  90% 
confidence  intervals  for  each  MOO/MOE  can  be  found  in  Appendix  B. 
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Table  8  shows  the  results  of  the  experiments  for  each  design  point  and  each 
MOO/MOE.  Analysis  of  each  MOO/MOE  follows. 


Table  8.  Design  Matrix  and  Mean  Responses  for  each  MOO/MOE 


FACTORS 

“  MOO/MOEs 

Design  Point 

Quality 

Quantity 

Timeliness 

Equipment 

Killed 

Equipment 
Killed  by 
Ground 

Equipment 
Killed  Per 
Mission 

Red/Blue  Air 
Losses 

%  Strat  Tgts 
Killed 

r~ 

-1 

-1 

-1 

7121.50 

6323.93 

i  rn 

1.23 

0.67 

2 

-1 

-1 

1 

7142.27 

6266.53 

59.42 

1.19 

0.67 

3 

-1 

1 

-1 

7209.23 

6085.30 

105.60 

1.62 

0.77 

4 

-1 

1 

1 

7253.63 

6092.73 

112.98 

1.57 

0.79 

5 

1 

-1 

-1 

7925.76 

7028.03 

108.01 

1.28 

0.80 

fl 

1 

-1 

1 

8131.47 

7399.70 

87.94 

1.15 

0.80 

7 

1 

1 

-1 

8140.70 

7265.13 

88.17 

1.53 

0.88 

8 

1 

1 

1 

8253.07 

7283.47 

89.22 

1.55 

0.88 

9 

-1 

0 

0 

7201.95 

6211.00 

103.95 

1.61 

0.73 

10 

1 

0 

0 

8269.70 

7245.00 

94.38 

1.55 

0.82 

11 

0 

-1 

0 

7809.60 

7172.00 

87.85 

1.33 

0.87 

12 

0 

1 

0 

7848.80 

6942.00 

111.78 

1.64 

0.85 

13 

0 

0 

-1 

7790.05 

7028.00 

94.23 

1.47 

0.85 

14 

0 

0 

1 

7823.60 

6913.00 

96.89 

1.58 

0.85 

15 

0 

0 

0 

7775.75 

6931.25 

104.98 

1.64 

0.89 

16 

0 

0 

0 

7926.25 

6995.50 

139.87 

1.68 

0.86 

17 

0 

0 

0 

7866.25 

6932.00 

153.59 

1.74 

0.86 

18 

0 

0 

0 

7834.75 

6916.00 

172.69 

1.63 

0.87 

19 

0 

0 

0 

7857.50 

6943.75 

128.60 

1.62 

0.87 

20 

0 

0 

0 

7836.50 

6903.00 

104.06 

1.47 

0.86 
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7.7.1  Total  Red  Equipment  Killed 

7.7.1.1  Screening  Experiment 

Table  9  and  Figure  34  summarize  the  results  of  the  screening  experiment  for  Total 
Red  Equipment  Killed.  Significant  factors  are  indicated  by  a  Prob  >  |t|  of  less  than  0.10 
in  Table  9.  Figure  34  graphically  shows  the  relative  sensitivity  of  the  main  factors.  For 
this  MOO/MOE,  the  significant  factors  are  Quality,  Quantity,  Timeliness  and  Quality2. 


Table  9.  Screening  Fit  for  Total  Red  Equipment  Killed 


Term 

Parameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

479.211 

16.330 

29.35 

<.0001 

Quantity 

57.484 

16.330 

3.52 

0.0055 

Timeliness 

41.679 

16.330 

2.55 

0.0288 

Quality*Quality 

-123.001 

31.140 

-3.95 

0.0027 

Quantity*Quality 

17.180 

18.258 

0.94 

0.3689 

Quantity*  Quantity 

-29.626 

31.140 

-0.95 

0.3638 

Timeliness*Quality 

31.614 

18.258 

1.73 

0.114 

T  imeliness*  Quantity 

-8.714 

18.258 

-0.48 

0.6434 

Timeliness*Timeliness 

-52.001 

31.140 

-1.67 

0.1259 

Quality  Quantity  Timeliness 


Figure  34.  Total  Red  Equipment  Killed  Prediction  Profile 
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7.7.1.2  Model 


The  response  model  is: 


y  =  7836.905  +  479.21 1  x  Quality  +  57.484  x  Quantity  + 
4 1 .679  x  Timeliness  - 1 71 .978  x  Quality2 


The  model  parameter  estimates  are  shown  in  Table  10. 


Table  10.  Total  Red  Equipment  Killed  Model  Parameter  Estimates 


Term 

Parameter  Estimates 

Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

7836.905 

18.760 

417.75 . . 

<.0001 

Quality 

479.211 

18.760 

25.54 

<.0001 

Quantity 

57.484 

18.760 

3.06 

0.0079 

Timeliness 

41.679 

18.760 

2.22 

0.0421 

Quality*  Quality 

-171.978 

26.531 

-6.48 

<0001 

The  R2Adj  for  the  model  is  high  (Table  1 1),  and  the  model  is  significant  (Table 


12). 


Table  11.  Total  Red  Equipment  Model  Summary  of  Fit 


RSquare 

0.97928 

RSquare  Adj 

0.97375 

Root  Mean  Square  Error 

59.3242 

Mean  of  Response 

7750.92 

Observations  (or  Sum  Wgts) 

20 

Table  12.  Total  Red  Equipment  Killed  Analysis  of  Variance 


Source 

- DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

4 

24947273  “ 

623682 

177.215 

Error 

15 

52790.4 

3519 

Prob>F 

C  Total 

19 

2547517.9 

<0001 
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7.7.1.3  Response  Surface 

The  response  surfaces  for  Total  Red  Equipment  Killed  are  seen  in  Figure  35. 
These  indicate  that  Total  Red  Equipment  Killed  is  very  sensitive  to  the  Quality  factor. 
As  Quality  increases,  there  are  relatively  large  increases  in  Total  Red  Equipment  killed. 

Timeliness  =  0 


Figure  35.  Response  Surfaces  for  Total  Red  Equipment  Killed  Model 
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The  Quantity  factor  plays  a  slight  role  in  increasing  the  number  of  equipment  killed  as  it 
moves  to  the  “high”  setting.  Timeliness  has  even  less  of  an  effect  than  Quantity. 

7.7.1.4  Significant  Differences 

Table  13  indicates  significant  differences  in  mean/median  response  among  the 
first  eight  design  points.  Since  the  table  is  symmetric,  only  the  upper  half  has  been  filled. 
Referring  back  to  Equation  (3),  Design  Points  1-4  have  Quality  set  at  “low”,  and  Design 
Points  5-8  have  Quality  set  at  “high”.  Whenever  Quality  changes  from  “low”  to  “high”, 
there  is  a  significant  difference,  as  is  evidenced  by  the  upper  right  16  blocks.  Within 
Design  Points  1-4,  there  is  a  difference  when  Quality  changes.  Within  Design  Points  5-8, 
differences  mainly  exist  with  Design  Point  8.  Referring  to  Appendix  B,  we  see  that  the 
differences  within  Design  Points  1-4  and  5-8  are  an  order  of  magnitude  lower  than  the 
differences  between  the  groups.  This  indicates,  and  confirms,  that  Quality  is  the  most 
important  factor  for  Total  Red  Equipment  Killed. 


H=  Significant  Difference 

= No  Significant  Difference 
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7.7.2  Red  Equipment  Killed  in  Ground  Battle 
7.7.2.1  Screening  Experiment 

The  screening  experiment  for  Red  Equipment  Killed  in  Ground  Battle,  shown  in 
Table  14  and  Figure  36,  calculated  the  significant  factors  to  be  Quality,  Quantity, 
Timeliness,  Quality2,  Quantity* Quality,  and  Timeliness*Quality.  Note  that  Timeliness 
was  not  significant  by  itself,  but  was  included  because  the  interaction  term, 

Timeliness*  Quality,  was  significant. 

Table  14.  Screening  Fit  for  Red  Equipment  Killed  in  Ground  Battle 


Term 

Parameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

HI 

Quality 

524.183 

25.428 

20.61 

<.0001 

Quantity 

-52.157 

25.428 

-2.05 

0.0674 

Timeliness 

22.503 

25.428 

0.88 

0.3969 

Quality*Quality 

-268.394 

48.489 

-5.54 

0.0002 

Quantity*Quality 

66.663 

28.429 

2.34 

0.041 

Quantity*  Quantity 

60.606 

48.489 

1.25 

0.2398 

Timeliness*Quality 

54.996 

28.429 

1.93 

0.0818 

Timeliness*  Quantity 

-36.063 

28.429 

-1.27 

0.2334 

Timeliness*  Timeliness 

-25.894 

48.489 

-0.53 

0.605 

7399.7 

<n 

*  6960.708 

c 

&  6085.3 


Quality  Quantity  Timeliness 

Figure  36.  Red  Equipment  Killed  in  Ground  Battle  Prediction  Profile 
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1.12.2  Model 


The  response  model  is: 


y  =  6967.65  +  524.183  x  Quality  -  52.157  x  Quantity  +  22.503  x  Timeliness 
-  247.567  x  Quality 2  +  66.663  x  Quality  x  Quantity 
+  54.996  x  Quality  x  Timeliness 


The  model  parameters  are  shown  in  Table  15. 


Table  15.  Red  Equipment  Killed  in  Ground  Battle  Parameter  Estimates 


Term 

Parameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

6967.650 

25.600 

272.18 

<.0001 

Quality 

524.183 

25.600 

20.48 

<.0001 

Quality*  Quality 

-247.567 

36.204 

-6.84 

<.0001 

Quantity 

-52.157 

25.600 

-2.04 

0.0625 

Timeliness 

22.503 

25.600 

0.88 

0.3953 

Quality*  Quantity 

66.663 

28.621 

2.33 

0.0366 

Quality*Timeliness 

54.996 

28.621 

1.92 

0.0769 

The  R2Adj  (Table  16)  is  high,  and  the  model  is  significant  as  seen  in  Table  17. 

Table  16.  Red  Equipment  Killed  in  Ground  Battle  Model  Summary  of  Fit 


RSquare 

0.97364 

RSquare  Adj 

0.96147 

Root  Mean  Square  Error 

80.9536 

Mean  of  Response 

6843.87 

Observations  (or  Sum  Wgts) 

20 

Table  17.  Red  Equipment  Killed  in  Ground  Battle  Analysis  of  Variance 


Source 

— nr- 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

- 6 — 

3146142.6 

524357 

80.0119 

Error 

13 

85195.3 

6553 

Prob>F 

C  Total 

19 

3231337.9 

<.0001 

(18) 
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1.1.23  Response  Surface 

The  response  surfaces  for  Red  Equipment  Killed  in  Ground  Battle  are  shown  in 
Figure  37.  The  Quality  factor  is  again  the  most  influential  on  the  model  response,  with 
Quantity  at  a  distant  second.  When  Quality  is  set  at  “low”,  Equipment  Killed  actually 
decreases  slightly  as  Quantity  increases. 


Timeliness  =  0 


Low  Low  Quality 


Timeliness  =  -1 


Timeliness  =  1 


Figure  37.  Response  Surfaces  for  Red  Equipment  Killed  in  Ground  Battle  Model 
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7.7.2.4  Significant  Differences 


Table  18  indicates  if  there  is  a  significant  difference  in  mean/median  response 
among  the  first  eight  design  points.  Again,  there  are  always  differences  when  Quality 
moves  from  “low”  to  “high”.  No  differences  exist  when  only  Timeliness  changes. 

Table  18.  Significant  Differences  in  Red  Equipment  Killed  in  Ground  Battle 

Design  Point  I  1  3  3  5  f>  7  8 

1 
2 

3 

4 

5 

6 

7 

8 


=  Significant  Difference 
= No  Significant  Difference 

7.7.3  Red  Equipment  Killed  by  Air  Missions 
7.7.3.1  Screening  Experiment 

The  following  factors  are  significant  for  Red  Equipment  Killed  by  Air  Missions 
as  shown  in  Table  19  and  Figure  38:  Quality,  Quantity,  Timeliness,  Quality2,  Quantity2, 
and  Quantity  *  Quality. 
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Table  19.  Screening  Fit  for  Red  Equipment  Killed  by  Air  Missions 


- Pi 

Term 

irameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

914.920 

12.499 

7J1 

<.0001 

Quality 

-87.150 

11.497 

-7.58 

<.0001 

Quantity 

124.880 

11.497 

10.86 

<.0001 

Timeliness 

31.505 

11.497 

2.74 

0.0208 

Quality*  Quality 

70.650 

21.924 

3.22 

0.0091 

Quantity*  Quality 

-25.731 

12.854 

-2 

0.0732 

Quantity*  Quantity 

-48.850 

21.924 

-2.23 

0.05 

Timeliness*Quality 

-2.394 

12.854 

-0.19 

0.856 

Timeliness*  Quantity 

1.556 

12.854 

0.12 

0.906 

T  imeliness*  T  imeliness 

-18.375 

21.924 

-0.84 

0.4215 

5 
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Figure  38.  Red  Equipment  Killed  by  Air  Missions  Prediction  Profile 


7.73.2  Model 

The  response  model  is: 

y  =  912.623  -  87. 1 50  x  Quality  + 124.880  x  Quantity  +  3 1 .505  x  Timeliness 
+  63.759  x  Quality 2  -  55.741  x  Quantity 2  -  25.73 1  x  Quantity  x  Quality 

The  model  parameter  estimates  are  summarized  in  Table  20. 


(19) 
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Table  20.  Red  Equipment  Killed  by  Air  Missions  Parameter  Estimates 


Parameter  Estimates 

Term 

Estimate 

Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

912.623 

11.090 

82.29  ' 

<.0001 

Quality 

-87.150 

10.456 

-8.34 

<.0001 

Quantity 

124.880 

10.456 

11.94 

<.0001 

Timeliness 

31.505 

10.456 

3.01 

0.01 

Quality*  Quality 

63.759 

18.484 

3.45 

0.0043 

Quantity*  Quality 

-25.731 

11.690 

-2.2 

0.0464 

Quantity*  Quantity 

-55.741 

18.484 

-3.02 

0.0099 

The  R2Adj  is  fairly  high  for  this  model  as  seen  in  Table  21 .  The  model  is 
significant  as  can  be  seen  in  Table  22. 


Table  21.  Red  Equipment  Killed  by  Air  Missions  Model  Summary  of  Fit 


RSquare 

RSquare  Adj 

0.92471 

Root  Mean  Square  Error 

33.0644 

Mean  of  Response 

916.633 

Observations  (or  Sum  Wgts) 

20 

Table  22.  Red  Equipment  Killed  by  Air  Missions  Analysis  of  Variance 


Source 

- DF 

Sum  of  Squares 

Mean  Square 

B1SS9I 

mm 

Error 

14212.33 

1093.3 

Prob>F 

C  Total 

Kfl 

275873.58 

<.0001 

7.7.3.3  Response  Surface 

The  response  surfaces  for  Red  Equipment  Killed  by  Air  Missions  are  seen  in 
Figure  39.  Unlike  previous  models,  Quantity  has  the  biggest  effect  on  the  responses, 
with  Quality  also  causing  changes.  Timelines  has  a  small  impact  as  evidenced  by  the 
surface  shifting  down  when  it  is  set  to  “low”. 
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Timeliness  =  0 


High 

Quantity 

Low  Low  Quality 


Timeliness  =  -1 


Timeliness  =  1 


High 

Quantity 


Low  Low  Quality 


Figure  39.  Response  Surfaces  for  Red  Equipment  Killed  by  Air  Missions 

7.7.3.4  Significant  Differences 

Table  23  indicates  if  there  is  a  significant  difference  in  mean/median  response 
among  the  first  eight  design  points.  Design  Point  pairs  3  &  4  ,  and  5  &  6  differ  only  by 
Timeliness.  This  MOE  has  the  most  significant  differences  among  the  Design  Points. 
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Table  23.  Significant  Differences  in  Red  Equipment  Killed  by  Air  Missions 


H=  Significant  Difference 

= No  Significant  Difference 


7.7.4  Air  Loss  Exchange  Ratio 
7.7.4.1  Screening  Experiment 

Table  24  and  Figure  40  summarize  the  significant  factors  for  Air  Loss  Exchange 
Ratio  found  through  the  screening  experiment  to  be  Quantity,  Timeliness,  Quantity2,  and 
Timeliness2.  Timeliness  was  included  because  Timeliness2  was  significant. 


Table  24.  Screening  Fit  for  Air  Loss  Exchange  Ratio 


|  Parameter  Estimates 

Term 

Estimate 

Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

mwMm 

0.026 

<.0001 

Quality 

-0.016 

0.024 

-0.67 

Quantity 

0.173 

0.024 

7.3 

Timeliness 

0.024 

-0.42 

Quality*  Quality 

0.045 

-0.64 

Quantity*  Quality 

-0.014 

0.026 

-0.54 

0.6016 

Quantity*Quantity 

-0.122 

0.045 

-2.71 

0.0221 

Timeliness*  Quality 

-0.001 

0.026 

-0.05 

0.9625 

Timeliness*  Quantity 

0.016 

0.026 

0.61 

0.5529 

Timeliness*Timeliness 

-0.081 

0.045 

-1.8 

0.1014 
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Figure  40.  Air  Loss  Exchange  Ratio  Prediction  Profile 


7.1 .4.2  Model 


The  response  model  is: 

y  =  1.617  +  0.173x  Quantity  —  0.01 0  x  Timeliness  —  0. 1 33  x  Quantity 2 
-  0.092  x  Timeliness 2 

The  model  parameters  are  shown  in  Table  25. 


Table  25.  Air  Loss  Exchange  Ratio  Model  Parameter  Estimates 


Parameter  Estimates 

Term 

Estimate 

Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

1.617 

0.02  M" 

73.55 

<.0001 

Quantity 

0.173 

0.02072 

8.33 

<0001 

Timeliness 

-0.010 

0.02072 

-0.48 

0.6351 

Quantity*  Quantity 

-0.133 

0.03663 

-3.62 

0.0025 

Timeliness*Timeliness 

-0.092 

0.03663 

-2.51 

0.0238 

0  7  0  T  0 

Quality  Quantity  Timeliness 


The  R2Adj  (Table  26)  is  fairly  high,  and  the  model  is  significant  (Table  27). 
Table  26.  Air  Loss  Exchange  Ratio  Model  Summary  of  Fit 


RSquare 

"1TH8646 

RSquaxe  Adj 

0.85618 

Root  Mean  Square  Error 

0.06553 

Mean  of  Response 

1.50412 

Observations  (or  Sum  Wgts) 

20 

(20) 
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Table  27.  Air  Loss  Exchange  Ratio  Analysis  of  Variance 


Source 

— DF 

Sum  of  Squares 

Mean  Square 

4 

Error 

15 

0.064 

0.004 

Prob>F 

C  Total 

19 

0.567 

<.0001 

7.1.43  Response  Surface 

The  response  surfaces  for  the  Air  Loss  Exchange  Ratio  are  seen  in  Figure  41. 
Note  that  the  axes  are  Quantity  and  Timeliness  since  Quality  is  not  in  the  model. 
Quantity  has  the  largest  effect  on  the  response.  The  highest  response  is  when  Timeliness 
is  near  the  “Center”.  Quality  has  no  effect. 

7.7.4.4  Significant  Differences 

Table  28  indicates  if  there  is  a  significant  difference  in  mean/median  response 
among  the  first  eight  design  points.  This  MOE  has  the  fewest  number  of  significant 
differences  among  Design  Points.  In  fact,  except  for  the  pair  3  &  8,  the  only  significant 
differences  are  present  when  Quantity  is  different  between  the  Design  Points.  Referring 
to  Appendix  B,  the  difference  between  3  &  8  is  barely  significant,  with  a  lower  bound  at 
0.02. 
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Figure  41.  Response  Surfaces  for  Air  Loss  Exchange  Ratio  Model 
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Table  28.  Significant  Differences  in  Air  Loss  Exchange  Ratio 


H=  Significant  Difference 

= No  Significant  Difference 


7.7.5  Percentage  of  Red  Strategic  Targets  Killed 
7.7.5.1  Screening  Experiment 

The  significant  factors  for  the  Percentage  of  Red  Strategic  Targets  Killed  are 
Quality,  Quantity,  and  Quality2  (shown  in  Table  29  and  Figure  42). 


Table  29.  Screening  Fit  for  Percentage  of  Red  Strategic  Targets  Killed 


Par 

Term  | 

ameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

0.863 

103.06 

<.0001 

0.056 

7.21 

<.0001 

Quantity 

0.036 

0.008 

4.64 

0.0009 

Timeliness 

0.002 

0.008 

0.21 

0.8358 

Quality*  Quality 

-0.082 

0.015 

-5.61 

0.0002 

Quantity*  Quality 

-0.007 

0.009 

-0.87 

0.4049 

Quantity*Quantity 

0.006 

0.015 

0.42 

0.6814 

Timeliness*  Quality 

-0.001 

0.009 

-0.11 

0.9182 

Timeliness*  Quantity 

0.001 

0.009 

0.11 

0.9122 

Timeliness*Timeliness 

-0.006 

0.015 

-0.4 

0.6973 
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Figure  42.  Percentage  of  Strategic  Targets  Killed  Prediction  Profile 


7J.5.2  Model 

The  response  model  is: 

y  -  0.864  +  0.056  x  Quality  +  0.036  x  Quantity  -  0.082  x  Quality2  (21) 


The  model  parameter  estimates  are  shown  in  Table  30. 

Table  30.  Percentage  of  Strategic  Targets  Killed  Model  Parameter  Estimates 


Term 

Parameter  Estimates 
Estimate  Std  Err 

t  Ratio 

Prob  >  |t| 

Intercept 

0.864 

0.006 

134.69 

<.OO01 

Quality 

0.056 

0.006 

8.67 

<0001 

Quality*  Quality 

-0.082 

0.009 

-9.08 

<0001 

Quantity 

0.036 

0.006 

5.58 

<0001 

The  R2Adj  (Table  31)  is  fairly  high,  and  the  model  is  significant  as  seen  in  Table 


32. 


Table  31.  Percentage  of  Red  Strategic  Targets  Killed  Model  Summary  of  Fit 


RSquare 

RSquare  Adj 

0.90717 

Root  Mean  Square  Error 

0.02027 

Mean  of  Response 

0.82236 
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Table  32.  Percentage  of  Red  Strategic  Targets  Killed  Analysis  of  Variance 


Source 

- DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

3 

62. #883 

Error 

16 

0.000 

Prob>F 

C  Total 

19 

7.7.5.3  Response  Surface 

The  response  surfaces  for  Percentage  of  Red  Strategic  Targets  Killed  are  seen  in 
Figure  43.  Quality  has  the  largest  effect,  although  Quantity  also  causes  a  change  in 
response. 

7.7.5.4  Significant  Differences 

Table  33  indicates  if  there  are  significant  differences  in  mean/median  response 
among  the  first  eight  design  points.  Since  Timeliness  has  no  impact,  there  are  no 
significant  differences  when  only  Timeliness  is  changed.  This  is  evidenced  in  Design 
Point  pairs  1  &  2,  3  &  4,  5  &  6,  and  7  &  8. 
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Timeliness  =  0 


Low  Low  Quality 


Timeliness  =  -1 


Timeliness  =  1 


High 

-0.7 

High  Quantity 


Low  Low  Quality 


Low  Low  Quality 


Figure  43.  Response  Surfaces  for  Percentage  of  Red  Strategic  Targets  Killed  Model 
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Table  33.  Significant  Differences  in  Percentage  of  Red  Strategic  Targets  Killed 


H=  Significant  Difference 

=  No  Significant  Difference 


7.8  Summary  and  Conclusion 

Table  34  summarizes  which  variables  were  considered  significant  in  each  of  the 
MOO/MOEs,  as  well  as  which  of  the  three  main  factors  -  Quality,  Quantity,  or 
Timeliness  -  was  the  most  influential.  Quality  and  Quantity  had  the  most  influence  on 
the  selected  MOO/MOEs.  Timeliness  did  not  play  a  very  significant  role.  This  is  not 
overly  surprising  since  Timeliness  is  represented  by  only  one  parameter,  and  if 
observations  are  delayed,  it  is  at  most  for  two  planning  cycles. 

Table  35  summarizes  the  significant  differences  in  mean/median  response  among 
the  first  eight  design  points.  The  number  in  each  square  is  the  number  of  times  the  two 
design  points  were  significantly  different,  across  the  five  MOO/MOEs.  From  this  table 
we  see  that  Design  Point  pairs  land  2,  3  and  4,  5  and  6,  and  7  and  8  were  rarely 
significantly  different  in  response.  Referring  back  to  the  design  matrix  in  Equation  (3), 
these  pairs  only  differ  in  the  Timeliness  parameter,  again  confirming  that  Timeliness  did 
not  play  a  significant  role  in  battle  outcomes. 
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Table  34.  Summary  of  Significant  Factors  Over  All  MOO/MOEs 


MOO/MOEs 

Parameters 

Equipment  Killed 

Equipment  Killed 
by  Ground 

Equipment  Killed 
by  Air 

Air  Loss 
Exchange  Ratio 

%  Strat  Tgts 
Killed 

Quality 

X 

Quantity 

X 

X 

X 

X 

X 

X 

X 

X 

X 

£  . 

X 

X 

X 

x 

X 

X 

Timeliness*Timeliness 

X 

X  =  Included  in  the  Model 

XXX  =  Most  Influential  among  Quality,  Quantity,  Timeliness 


Based  on  the  results  shown  in  this  chapter,  the  battle  outcomes  are  responsive  to 
changes  in  the  ISR  parameters  in  THUNDER,  with  Quality  and  Quantity  parameters 
causing  the  most  responsiveness.  This  confirms  that  THUNDER  can  be  used  as  a  tool 
for  comparative  analysis  between  competing  ISR  systems. 


8.  Conclusions  and  Recommendations 


8.1  Validation  Assessment  of  THUNDER’S  ISR  Module 

THUNDER  is  a  vital  tool  for  Air  Force  acquisition  and  operational  planning 
decisions.  With  the  ever-increasing  impact  of  intelligence  on  information  superiority  and 
battlefield  success,  THUNDER  must  represent  the  ISR  aspect  of  combat  with  a  high 
degree  of  fidelity.  To  neglect  this  aspect  of  THUNDER  would  result  in  inaccurate 
assessments  and  false  assumptions  about  ISR  systems  and  their  ability  to  influence 
combat  outcomes. 

The  comparison  between  the  real  world  process  and  THUNDER’S  ISR  process 
presented  in  this  study  indicates  that  THUNDER’S  ISR  module  has  implemented  the  ISR 
process  with  a  considerable  degree  of  fidelity,  although  improvements  can  be  made. 
Although  the  processes  do  involve  some  aggregation  and  rely  heavily  on  user  input,  they 
do  reasonably  represent  the  real  world  processes,  especially  in  regard  to  the  air  war. 
THUNDER  does  not  capture  abstract  information  and  the  human  decision-making 
process.  It  focuses  more  on  winning  the  battle  instead  of  winning  the  war.  Strategic  uses 
of  ISR  such  as  deciding  when  to  strike  and  where  to  attack  are  lost.  It  is  possible  to 
artificially  capture  some  of  these  aspects  through  user  input,  but  that  makes  the  already 
user-intensive  input  process  even  more  laborious,  and  somewhat  defeats  the  purpose  of 
having  a  model  to  enact  the  process.  These  elements  are  often  neglected  in  campaign- 
level  models,  and  although  they  are  extremely  important  issues  in  the  real  world,  they  are 
not  a  major  degradation  of  the  validity  of  THUNDER’S  ISR  module. 
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The  experiment  performed  in  this  study  demonstrated  that  the  battle  outcomes  of 
THUNDER  reflect  the  importance  of  ISR  capabilities  through  their  responsiveness  to 
differences  in  the  quality,  quantity,  and  timeliness  of  ISR.  This  validates  THUNDER’S 
use  for  comparative  analysis  between  competing  ISR  systems.  It  also  demonstrates  the 
compounding  effects  of  ISR  in  THUNDER.  Poor  ISR  results  in  poor  prioritization  of 
targets  and  poor  aircraft/munition  selection,  which  results  in  low  mission  effectiveness. 
Low  mission  effectiveness  results  in  fewer  targets  being  hit,  more  missions  needed,  more 
Blue  targets  being  hit,  more  aircraft  endangered,  etc.  Because  THUNDER  reflects  these 
compounding  effects,  it  is  that  much  more  important  that  the  processes  implemented  in 
THUNDER  closely  represent  the  real  world  process. 

The  importance  of  user  input  in  THUNDER  cannot  be  overemphasized.  The  real 
flexibility,  and  some  of  the  resulting  validity,  of  the  ISR  module  comes  from  the  fact  that 
the  user  can  script  or  influence  so  many  different  aspects  to  capture  ISR  effects.  The 
capabilities  of  the  sensors,  the  degradations  of  their  observations  over  time,  and  the  target 
prioritization  rules  must  be  carefully  input  as  the  confidence  levels  and  perception  levels 
of  the  sensors,  and  the  rules  set  for  target  nomination,  greatly  influence  the  perceptions  of 
targets  and  targeting  decisions. 

8.2  Recommendations 

Improvements  are  possible  to  increase  the  precision  of  THUNDER’S  ISR  module. 
The  goal  of  improving  THUNDER’S  ISR  module  is  not  to  make  it  an  explicit,  high 
resolution  module,  but  the  following  suggestions  should  help  to  improve  the  fidelity  of 
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the  model  bearing  in  mind  the  purpose  of  campaign-level  models.  Recommendations  for 
improvement  are: 

-  Coordinate  among  air,  satellite,  and  scripted  events  to  avoid  unnecessary 
redundancy  and/or  to  ensure  desired  redundancy. 

—  Planning  must  consider  availability  of  assets. 

-  Aid  in  determining  ground  unit  posture  through  adjustment  on  force  ratio. 
Allow  unit  to  attack,  or  not  to  attack,  based  on  perception  of  enemy’s  support 
status. 

Adjust  fusion  algorithm  to  incorporate  previous  knowledge  and  other 
observations  available. 

Consider  the  confidence  level  of  the  sensor  performing  BDA  for  the  target 
perception  and  target  nomination  rules. 

-  Allow  Grid  and  FLOT  sensors  to  impact  the  ground  war. 

-  Allow  aircraft  to  carry  more  than  one  sensor.  Also,  more  than  one  sensor  can 
be  available  for  all  ISR  assets  with  the  best  one  or  two  selected  depending  on 
target  type. 

A  final  recommendation  is  to  ensure  that  the  results  found  in  the  experiment 
conducted  in  this  study  are  consistent  with  results  that  would  be  found  using  classified,  or 
more  realistic  data. 


8.3  Final  Thought 

THUNDER’S  ISR  module  is  definitely  a  useful  and  necessary  element.  The  ISR 
module  has  evolved  over  the  past  few  years  and  is  continually  being  improved.  The 
recommendations  presented  above  will  only  enhance  THUNDER’S  utility  and  capability 
beyond  that  which  it  has  already. 
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Appendix  A:  Glossary  of  Acronyms 


ACC 

Air  Combat  Command 

AF 

Air  Force 

AFSAA 

Air  Force  Studies  and  Analyses  Agency 

AFSATCOM 

Air  Force  Satellite  Communications 

ASAS 

All-Source  Analysis  System 

ATARS 

Advanced  Tactical  Air  Reconnaissance  Systems 

ATCAL 

Attrition  Calibration 

ATO 

Air  Tasking  Order 

AUV 

Autonomous  Underwater  Vehicles 

AWACS 

Airborne  Warning  and  Control  System 

BAI 

Battlefield  Air  Interdiction 

BDA 

Battle  Damage  Assessment 

C4I 

Command,  Control,  Communications,  Computer,  and  Intelligence 

CAS 

Close  Air  Support 

CASCOM 

Combined  Arms  Support  Command 

CCD 

Central  Composite  Design 

CEM 

Concept  Evaluation  Model 

CHATS 

CI/HUMINT  Automated  Tool  Set 

Cl 

Counterintelligence 

CIA 

Central  Intelligence  Agency 

CINC 

Commander-in-Chief 

CNES 

Centre  National  d’Etudes  Spatiales 

COMINT 

Communications  Intelligence 

COMSEC 

Communications  Security 

COSAGE 

Concepts  and  Analysis  Agency’s  Combat  Sample  Generator 

COSPO 

Community  Open  Source  Program  Office 

DAMA 

Demand  Assigned  Multiple  Access 

DEA 

Drug  Enforcement  Agency 

DIA 

Defense  Intelligence  Agency 

DIDS 

Defense  Intelligence  Dissemination  System 

DISN 

Defense  Information  System  Network 

DO 

Directorate  for  Intelligence  Operations 

DoD 

Department  of  Defense 

DoDIIS 

DoD  Intelligence  Information  System 

DSCS 

Defense  Satellite  Communications  System 

DSP 

Defense  Support  Program 
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EA 

ECM 

EHF 

ELINT 

ENSCE 

ESAMS 

FAX 

FBI 

FISINT 

FLOT 

GBCS 

GBS 

GCCS 

GPS 

HQ 

HUMINT 

ICBM 

IDHS 

IMINT 

INF 

INFOSEC 

INMARSAT 

INSCOM 

IPB 

INTELSAT 

ISR 

JAOC 

JC2WC 

JDISS 

JFACC 

JFC 

JFLCC 

JFSOCC 

JIC 

JISE 

JSIPS 

J-STARS 

JTF 

JWICS 


Electronic  Attack 

Electronic  Countermeasures 

Extremely-High  Frequency 

Electronic  Intelligence 

Enemy  Situation  Correlation  Element 

Enhanced  Surface  to  Air  Missile  Simulation 

Facsimile 

Federal  Bureau  of  Investigation 

Foreign  Instrumentation  Signals  Intelligence 

Forward  Line  of  Troops 

Ground-Based  Common  Sensor 
Global  Broadcast  Service 
Global  Command  and  Control  System 
Global  Positioning  System 

Headquarters 
Human  Intelligence 

Intercontinental  Ballistic  Missile 
Intelligence  Data  Handling  Systems 
Imagery  Intelligence 
Intermediate-Range  Nuclear  Forces 
Information  Security 
International  Maritime  Satellite 
Intelligence  and  Security  Command 
Intelligence  Preparation  of  the  Battlefield 
International  Telecommunications  Satellite 
Intelligence,  Surveillance,  and  Reconnaissance 

Joint  Air  Operations  Center 

Joint  Command  and  Control  Warfare  Center 

Joint  Deployable  Intelligence  Support  System 

Joint  Force  Air  Component  Commander 

Joint  Force  Commander 

Joint  Land  Component  Commander 

Joint  Force  Special  Ops  Component  Commander 

Joint  Intelligence  Center 

Joint  Intelligence  Support  Element 

Joint  Service  Imagery  Processing  System 

Joint  Surveillance  Target  Attack  Radar  System 

Joint  Task  Force 

Joint  Worldwide  Intelligence  Communications  System 
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MASINT 

Measurement  and  Signature  Intelligence 

MCIA 

Marine  Corps  Intelligence  Activity 

ME 

Middle  East 

MILSATCOM 

Military  Satellite  Communications 

MOE 

Measure  of  Effectiveness 

MOO 

Measure  of  Outcome 

MSE 

Mean  Square  Error 

MSR 

Mean  Square  Regression 

NAIC 

National  Air  Intelligence  Center 

NATO 

North  Atlantic  Treaty  Organization 

NBC 

Nuclear,  Biological,  Chemical 

NCA 

National  Command  Authority 

NIMA 

National  Imagery  and  Mapping  Agency 

NIST 

National  Intelligence  Support  Team 

NMIC 

National  Maritime  Intelligence  Center 

NMJIC 

National  Military  Joint  Intelligence  Center 

NRO 

National  Reconnaissance  Office 

NSA 

National  Security  Agency 

OPSEC 

Operations  Security 

OSINT 

Open  Source  Intelligence 

Pk 

Probability  of  Kill 

RADGUNS 

Radar-Directed  Gun  System  Simulation 

RADINT 

Radar  Intelligence 

RAF 

Royal  Air  Force 

RCS 

Radar  Cross  Section 

RECCE 

Reconnaissance 

S3I 

System  Simulation  Solutions,  Inc. 

SAAI 

Information  Superiority  Branch  (within  AFSAA) 

SAAW 

Wargaming  Branch  (within  AFSAA) 

SAM 

Surface-to-Air  Missile 

SAR 

Synthetic  Aperture  Radar 

SBIRS 

Space-Based  Infrared  System 

SCI 

Sensitive  Compartmented  Information 

SEAD 

Suppression  of  Enemy  Air  Defenses 

SHF 

Super-High  Frequency 

SIGINT 

Signals  Intelligence 

SLBM 

Submarine  Launched  Ballistic  Missile 

SOF 

Special  Operations  Forces 

SREC 

Standoff  Reconnaissance 

SSE 

Sum  of  Squares  for  Error 
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SSR 

Sum  of  Squares  for  Regression 

START 

Strategic  Arms  Reduction  Treaty 

TACS 

Tactical  Air  Control  System 

TADIX-B 

Tactical  Data  Information  Exchange  System-Broadcast 

TBM 

Tactical  Ballistic  Missile 

TDRSS 

Tracking  and  Data  Relay  Satellite  System 

TECHINT 

Technical  Intelligence 

TEG 

Tactical  Exploitation  Group 

TELINT 

Telemetry  Intelligence 

TIBS 

Tactical  Information  Broadcast  System 

TLAM 

Tomahawk  Land  Attack  Missile 

TRAP 

Tactical  Receive  Equipment  and  Related  Applications 

UAV 

Uninhabited  Aerial  Vehicle 

UHFF/O 

Ultra-High  Frequency  Follow-on 

UHF 

Ultra-High  Frequency 

US 

United  States 

USMC 

United  States  Marine  Corp 

USN 

United  States  Navy 
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Appendix  B:  Confidence  Interval  Summaries 


90%  Confidence  Intervals  for  Total  Red  Equipment  Killed 


Mean/Median  of  Design  Point  1  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

2 

3 

4 

5 

6 

7 

8 

Lower  Bound 

-126.46 

-196.72 

JgggJI 

-1049.00 

-1112.99 

-1118.94 

-1231.16 

Upper  Bound 

84.93 

21.25 

-70.00 

-755.00 

-906.94 

-919.46 

-1031.97 

Significant? 

No 

No 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  2  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

3 

4 

5 

6 

7 

8 

Lower  Bound 

-84.93 

-174.02 

-182.00 

-1019.00 

-1090.18 

-1095.99 

-1208.21 

Upper  Bound 

126.46 

40.09 

-43.00 

-737.00 

-888.22 

-900.87 

-1013.39 

Significant? 

No 

No 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  3  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

4 

5 

6 

7 

8 

Lower  Bound 

-21.25 

-40.09 

-107.00 

-967.00 

-1026.65 

-1032.65 

-1144.87 

Upper  Bound 

196.72 

174.02 

34.00 

-657.00 

-817.81 

gjgjgjl 

-942.79 

Significant? 

No 

No 

No 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  4  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

5 

6 

7 

8 

Lower  Bound 

70.00 

43.00 

-34.00 

-924.00 

-918.00 

-940.00 

-1057.00 

Upper  Bound 

213.00 

182.00 

107.00 

QjjgjjH 

-790.00 

-796.00 

-926.00 

Significant? 

Yes 

Yes 

No 

Yes 

Yes 

Mean/Median  of  Design  Point  5  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

6 

7 

8 

Lower  Bound 

755.00 

737.00 

657.00 

647.00 

-261.00 

-269.00 

-372.00 

Upper  Bound 

1049.00 

1019.00 

967.00 

924.00 

64.00 

35.00 

-77.00 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 
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Mean/Median  of  Design  Point  6  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

7 

8 

Lower  Bound 

1112.99 

888.22 

817.81 

790.00 

-64.00 

-216.11 

Upper  Bound 

906.94 

1090.18 

1026.65 

918.00 

261.00 

85.43 

-27.09 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

No 

Mean/Median  of  Design  Point  7  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

8 

Lower  Bound 

919.46 

900.87 

830.28 

796.00 

-35.00 

-85.43 

-203.21 

Upper  Bound 

1118.94 

1095.99 

1032.65 

940.00 

269.00 

103.90 

-21.52 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 

Mean/Median  of  Design  Point  8  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

7 

Lower  Bound 

1031.97 

926.00 

77.00 

27.09 

21.52 

Upper  Bound 

1231.16 

1208.21 

1144.87 

1057.00 

372.00 

216.11 

203.21 

Significant? 

— 

Yes 

Yes 

KUB 

Yes 

Yes 

Yes 

90%  Confidence  Intervals  for  Red  Equipment  Killed  in  Ground  Battle 


Mean/Median  of  Design  Point  1  -  Mean/Median  of  Design  Point  l 

Design  Point  i 

2 

3 

4 

5 

6 

7 

8 

Lower  Bound 

-42.35 

161.00 

158.00 

-1118.00 

-1175.43 

-1027.64 

-1051.32 

Upper  Bound 

157.15 

298.00 

286.00 

-799.00 

-976.10 

-854.76 

-867.75 

Significant? 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  2  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

3 

4 

5 

6 

7 

8 

Lower  Bound 

-157.15 

109.00 

106.00 

-1176.00 

-1230.91 

-1082.73 

-1106.63 

Upper  Bound 

42.35 

241.00 

234.00 

-876.00 

-1035.42 

-914.47 

-927.24 

Significant? 

No 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  3  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

4 

5 

6 

7 

8 

Lower  Bound 

-298.00 

-241.00 

-51.00 

-1344.00 

-1346.00 

-1204.00 

-1240.00 

Upper  Bound 

-161.00 

-109.00 

41.00 

-1091.00 

-1225.00 

-1089.00 

-1110.00 

Significant? 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  4  -  Mean/Median  ot  Design  t'oint  i 

Design  Point  i 

1 

2 

3 

5 

6 

7 

8 

Lower  Bound 

-286.00 

-234.00 

-41.00 

-1340.00 

-1336.00 

-1192.00 

-1223.00 

Upper  Bound 

-158.00 

-106.00 

51.00 

-1088.00 

-1216.00 

-1082.00 

-1105.00 

Significant? 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  5  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

6 

7 

8 

Lower  Bound 

799.00 

876.00 

1091.00 

1088.00 

-293.00 

-100.00 

-135.00 

Upper  Bound 

1118.00 

1176.00 

1344.00 

1340.00 

46.00 

186.00 

167.00 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

Mean/Median  of  Design  Point  6  -  Mean/Median  of  Design  Point  l 

Design  Point  i 

1 

2 

3 

4 

5 

7 

8 

Lower  Bound 

1175.43 

1035.42 

1225.00 

1216.00 

-46.00 

50.54 

26.64 

Upper  Bound 

976.10 

1230.91 

1346.00 

1336.00 

293.00 

218.59 

205.83 

Significant? 

Yes 

■SB 

WtCFW 

mm 

No 

Yes 

Yes 
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Mean/Median  of  Design  Point  7  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

8 

Lower  Bound 

854.76 

914.47 

1089.00 

1082.00 

1082.00 

-218.59 

-92.51 

Upper  Bound 

1027.64 

1082.73 

1204.00 

1192.00 

-186.00 

-50.54 

55.85 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 

Mean/Median  of  Design  Point  8  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

7 

Lower  Bound 

867.75 

927.24 

1110.00 

1105.00 

-167.00 

-205.83 

-55.85 

Upper  Bound 

1051.32 

1106.63 

1240.00 

1223.00 

135.00 

-26.64 

92.51 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

No 
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90%  Confidence  Intervals  for  Red  Equipment  Killed  by  Air  Missions 


Mean/Median  of  Design  Point  I  -  Mean/Median  of  Design  Point  i 


Design  Point  1 


Lower  Bound 


Upper  Bound 


Significant? 


2  3  4  5  6 _ 7 _ 8 _ 


-154.32  -397.94  -426.21  12.03  10.68  -145.34  -251.09 


-47.35  -295.33  -307.06  122.91  115.86  -33.02  -142.07 


Yes  Yes  Yes  Yes  Yes  Yes  Yes 


Mean/Median  of  Design  Point  2  -  Mean/Median  of  Design  Point  l 


Design  Point  i 


Lower  Bound 

47.35 

-291.20 

-320.52 

118.15 

117.21 

-39.29 

-144.87 

Upper  Bound 

154.32 

-200.40 

-211.08 

218.45 

210.99 

62.59 

-46.63 

Significant? 

Yes 

■■ 

Yes 

Yes 

Yes 

No 

Yes 

Mean/Median  of  Design  Point  3  -  Mean/Median  of  Design  Point  l 


Design  Point  i 


Lower  Bound 


Upper  Bound 


Significant? 


295.33 


397.94 


Yes 


200.40 


291.20 


Yes 


-72.60 


32.60 


No 


5 


366.33 


461.87 


Yes 


6 


365.60 


454.20 


Yes 


7 


208.85 


306.05 


Yes 


8 


103.40 


196.70 


Yes 


Mean/Median  of  Design  Point  4  -  Mean/Median  of  Design  Point  i 


Design  Point  i  1  2  3  5  6 


Lower  Bound  307.06  211.08  -32.60  377.47  376.05  220.11 


Upper  Bound  426.21  320.52  72.60  490.73  483.75  334.79 


Significant?  Yes  Yes  No  Yes  Yes  Yes 


8 


114.29 


225.81 


s 


Mean/Median  of  Design  Point  5  -  Mean/Median  of  Design  Point  l 


Design  Point  l 


Lower  Bound 


Upper  Bound 


Significant? 


-122.91 


-12.03 


Yes 


2 


-218.45 


-118.15 


Yes 


-366.33  -377.47 


Yes  Yes 


-53.35 

-209.68 

-315.33 

44.95 

-103.62 

-212.77 

No 

Yes 

Yes 

Mean/Median  of  Design  Point  6  -  Mean/Median  of  Design  Point  l 


Design  Point  i  1  2  3 _ 4 _ 5 _ 7 _ 8 


Lower  Bound  -10.68  -210.99  -454.20  -483.75  -44.95  -202.41  -307.95 


Upper  Bound  -115.86  -117.21  -365.60  -376.05  53.35  -102.49  -211.75 


Significant?  |  Yes  |  Yes  Yes  Yes  No  |  Yes  |  Yes 


Mean/Median  of  Design  Point  7  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

8 

Lower  Bound 

33.02 

-62.59 

-306.05 

-334.79 

103.62 

102.49 

-159.46 

Upper  Bound 

145.34 

39.29 

-208.85 

-220.11 

209.68 

202.41 

-55.34 

Significant? 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Mean/Median  of  Design  Point  8  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

7 

Lower  Bound 

142.07 

46.63 

BEKirfil 

-225.81 

212.77 

211.75 

55.34 

Upper  Bound 

251.09 

144.87 

-103.40 

-114.29 

315.33 

307.95 

159.46 

Significant? 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 
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90%  Confidence  Intervals  for  Air  Loss  Exchange  Ratio 


Mean/Median  of  Design  Point  1  -  Mean/Median  of  Design  Point 


Design  Point  1 


Lower  Bound 

-0.09 

-0.48 

-0.47 

-0.15 

-0.03 

-0.44 

-0.38 

Upper  Bound 

■SB 

-0.26 

0.09 

0.16 

-0.22 

-0.16 

Significant? 

No 

Yes 

Yes 

No 

No 

Yes 

Yes 

Mean/Median  of  Design  Point  2  -  Mean/Median  of  Design  Point  i 


Design  Point  l 


Lower  Bound 


Upper  Bound 


Significant? 


-0.12 

-0.51 

msm 

-0.10 

■SB 

mg 

0.09 

-0.32 

-0.24 

0.06 

0.18 

-0.21 

-0.18 

No 

Yes 

Yes 

No 

No 

Yes 

Yes 

Mean/Median  of  Design  Point  3  -  Mean/Median  of  Design  Point  l 


Design  Point  i  1  2  4  5  6  7 


Lower  Bound  0.30  0.32  -0.08  0.24 


Upper  Bound  0.48  0.51  0.12  0.47  0.54 


Significant?  Yes  Yes  No  Yes  Yes  No 


Mean/Median  of  Design  Point  4  -  Mean/Median  of  Design  Point  i 


2  3  5  6  7  8 


-0.10  -0.02 


7  0.21 


No  No 


Design  Point  i 

1 

2 

3 

Lower  Bound 

0.26 

0.24 

-0.12 

Upper  Bound 

0.47 

0.51 

0.08 

Significant? 

Yes 

Yes 

No 

Mean/Median  of  Design  Point  5  -  Mean/Median  of  Design  Point  l 


Design  Point  i  1  2  3  4  6  7 


Lower  Bound  -0.09  -0.06  -0.47  -0.45  -0.02 


Upper  Bound  0.15  0.19  -0.24  -0.20  0.24 


Significant?  No  No  Yes  Yes  No 


Mean/Median  of  Design  Point  6  -  Mean/Median  of  Design  Point  i 


Design  Point  i 

1 

2 

3 

Lower  Bound 

0.03 

-0.18 

-0.54 

Upper  Bound 

-0.16 

0.10 

-0.35 

Significant? 

No 

No 

Yes 

-0.55 


-0.28 


-0.24 


0.02 


No 


Mean/Median  of  Design  Point  7  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

8 

Lower  Bound 

0.22 

0.21 

-0.17 

-0.17 

0.17 

0.24 

-0.05 

Upper  Bound 

0.44 

0.48 

■TtliEM 

0.10 

0.40 

0.52 

0.16 

Significant? 

Yes 

Yes 

No 

No 

Yes 

Yes 

No 

Mean/Median  of  Design  Point  8  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

7 

Lower  Bound 

0.16 

0.18 

miwxm 

-0.21 

0.12 

0.22 

msm 

Upper  Bound 

0.38 

0.40 

0.02 

0.44 

0.05 

Significant? 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

No 
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90%  Confidence  Intervals  for  Percentage  of  Red  Strategic  Targets  Killed 


Mean/Median  of  Design  Point  1  -  Mean/Median  of  Design  Point  i 


-0.14 


Design  Point  i 

2 

Lower  Bound 

-0.02 

Upper  Bound 

0.01 

Significant? 

No 

-0.23 

-0.23 

-0.20 

-0.21 

Yes 

Yes 

Mean/Median  of  Design  Point  2  -  Mean/Median  of  Design  Point  i 


Design  Point  i 


Lower  Bound 


Upper  Bound 


Significant? 


-0.12 

-0.14 

-0.07 

-0.08 

Yes 

Yes 

-0.12 


Yes 


7 


-0.23 


-0.19 


Yes 


Mean/Median  of  Design  Point  3  -  Mean/Median  of  Design  Point  i 


5  6  7 


-0.06 


2 


Yes 


Design  Point  i 

1 

2 

4 

5 

Lower  Bound 

0.08 

0.07 

-0.06 

Upper  Bound 

0.12 

0.12 

0.02 

0.01 

Significant? 

Yes 

Yes 

No 

No 

Mean/Median  of  Design  Point  4  -  Mean/Median  of  Design  Point  i 


-0.12 


6 


Yes 


Design  Point  i 

1 

2 

3 

5 

6 

Lower  Bound 

0.09 

0.08 

-0.02 

-0.05 

-0.05 

Upper  Bound 

0.14 

0.14 

0.05 

0.02 

0.00 

Significant? 

Yes 

Yes 

No 

No 

Yes 

Mean/Median  of  Design  Point  5  -  Mean/Median  of  Design  Point  i 


Design  Point  i 

1 

2 

3 

4 

Lower  Bound 

0.12 

0.11 

-0.02 

Upper  Bound 

0.16 

0.15 

0.06 

mss 

Significant? 

Yes 

Yes 

No 

No 

Mean/Median  of  Design  Point  6  -  Mean/Median  of  Design  Point  l 


2  3  4  5  7 


-0.09 


-0.06 


Yes 


Design  Point  i 

1 

2 

3 

4 

5 

Lower  Bound 

0.15 

0.12 

0.02 

0.00 

-0.02 

Upper  Bound 

0.12 

0.15 

0.06 

0.05 

0.02 

Significant? 

Yes 

Yes 

Yes 

Yes 

No 
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Mean/Median  of  Design  Point  7  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

8 

Lower  Bound 

0.20 

0.19 

0.08 

0.06 

0.05 

0.06 

-0.02 

Upper  Bound 

0.23 

0.23 

0.13 

0.12 

0.10 

0.09 

0.01 

Significant? 

Yes 

Yes 

mam 

Yes 

Yes 

Yes 

No 

Mean/Median  of  Design  Point  8  -  Mean/Median  of  Design  Point  i 

Design  Point  i 

1 

2 

3 

4 

5 

6 

7 

Lower  Bound 

0.21 

0.20 

B!&ub 

0.07 

0.06 

0.06 

Upper  Bound 

0.23 

0.23 

0.14 

0.13 

0.11 

0.10 

0.02 

Significant? 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 
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